Here are some observations that guide how I have been developing visual pipelines for Metaverse Arcologies.
Cloud City Manifesto
If the nation of the Internet had a capital, it would be Cloud City. Inhabited by a population the size of it’s network participants, built on substratum of data interfaces, blockchains and silicone circuits. A city in The Cloud, the internet reflected into metafiction, an open ledger of mythologies, where the dreamlines between synapse and symbol are blurred within a fluid of object interpreters. Here all cultures of the earth collide in a perceptual implosion of computer interfaces, data models and digital art.
Cloud City rejects no creed, invites all languages, and welcomes any inhabitants who come in peace. This is not your child safe metaverse, this is the burning jungle of human imagination, the frenetic pulse of the overheating GPUs, the rhythmic throbbing of a fresh bassline, an unfiltered interface to the substratum of network protocols.
Cloud City is a concept of the grown digital citizen who rejects the petty authoritarianism of walled gardens and is capable of representing themselves as a sovereign member of the internet.
1A. Universal Visual Language
Artifact – made of art. Early 19th century: from Latin arte ‘by or using art’ + factum ‘something made’ (neuter past participle of facere ‘make’).
Art is the language which transcends time and language barriers. Cultures have always been defined and remembered by their artifacts and narratives, their gods, their stories, artists, thinkers, musicians, and the mythologies they gave birth to. Art depicts particular cultural aesthetics, yet speaks a universally felt language and implicitly transfers symbolic meaning across cultural boundaries.
We remember humans and their cultures by means of Artifacts. Cultural artifacts are creative representations of the mythologies of which they were born, enacting the symbols and their meanings in a universal visual language.
This is where every culture of the ancient past has taken up it’s place in the living tapestry of history: in the world of Art.
“A skeuomorph is a derivative object that retains ornamental design cues from structures that were necessary in the original. ” – Wikipedia
When Apple first started designing mobile applications for the iPhone, they adopted the design principles of skeuomorphism. This is a digital interface that copies the appearance of its physical counterpart. The Compass app looked like the shape of a real metal compass with the wood and metal texture on it, and the Notepad app had a torn paper texture with horizontal lines on it just like real note paper.
Yet as soon as consciousness calibrated itself within the new virtual space, it rapidly adapted to what interface designs are best suited for the smartphone. Effective tooling and robustness soon became the primary design principle, and the old skeuomorphic interfaces were replaced by an abstract taxonomy of new interaction patterns that fit the emerging medium. Interface standards like Material UI on Android and iOS on Apple increasingly developed across native ecosystems to standardize out of skeuomorphism and into an abstract usability standard.
This process was progressed by legions of developers who strived to create more useful, practical and intuitive interfaces, and who all copied from each other what worked best and iterated on it. This transition from skeuomorphism to abstract interfaces appeared in software updates over the years which visually re-imagined computer interfaces from a physical portrait, and into a many faceted crystalline tool for navigating physical and virtual hyperspace.
Skeuomorphism served as a necessary growth phase to bridge from physical interfaces to abstract interfaces. At the beginning of it’s medium the best form of an abstract interface has not yet been discovered, so a physical interface is copied and iterated on, adapted for the medium until the optimal abstraction is discovered.
Post-skeuomorphism is a playful process of re-imagining computer interfaces starting with skeuomorphs and then rapidly iterating into abstract ends, natively adapted to the medium.
Metaverse Arcology is a post-skeuomorphic process of re-imagining content discoverability and surfing the open web in 3D space, using the skeuomorph of a virtual city as the departure point.
1C. Evolutionary Phase
“A factor of four is nothing to sneeze at.”
– John Carmack
The physical body has evolved for manipulating objects in 3D space, and yet through our obsession with screens, humans have retreated into 2D surface interfaces. It was so effective at achieving this that many humans have become fixated on screens.
Six Degrees of Freedom
At this time, most screen interfaces use 2D control surfaces, such as a mouse, tablet pen or fingers touching the surface of a screen. Since the invention of the iPhone we have close to perfected the 2D control surface, and yet it is still constrained to 2 dimensions.
Now a new type of input device is emerging that is 6 dimensional. This is now available in the new generation of interfaces widely distributed with VR and AR goggles. These Six Degrees of Freedom (6DOF) interfaces engage the entire body to interact with the computer in ways yet imagined.
It is called 6DOF because each hand has 3 axis of input for rotation, and 3 axis for location, totalling 6 axis. Multiply that by two hands, that is 12 axis of movement from the position of your hands. This marks an increase by a factor of 6 in terms of control axis over an old 2D control surface such as a touch screen.
With 6DOF, movement can be conducted as if doing tai chi while effectively using the computer interface. There is a new renaissance of interface designs emerging which engage the body much like in outer space. Astronauts remain an archetype of fascination, as the intrepid explorer exploring deeply into virtual space. Now interfaces are no longer constrained to a specific plane of interaction, and can be controlled in all directions arbitrarily.
For example, there are currently several types of tracking systems currently available as input and output devices for 6DOF interaction:
- Outside in (base stations) Valve Index, HTC Vive, Oculus Rift with Touch Controllers
- Inside out (HMD camera tracking) Oculus / Meta Quest, Vive Cosmos, Windows MR, Microsoft Hololens, Magic Leap, Nreal
- Computer vision (3D Hand tracking) Kinect, Webcam Motion Capture
An early form of skeuomorphism is currently seen in many VR goggle apps where you go into virtual space and you appear inside a virtual couch inside a virtual living room. Why is the furniture there if you can’t really sit on it? This is a classic incarnation of early phase skeuomorphism.
Apps such as Google Maps have perfected 2D navigation and now have nowhere to evolve but into augmented reality.
With augmented reality on the rise using passthrough video on VR goggles and a new generation of smart glasses, the practicality of skeuomorphism is fading, and a new standard of abstract interfaces is evolving, optimized for augmented reality.
These implement object pattern overlays create a new visual taxonomy to identify virtual objects from physical objects. The differentiation is informed to save people the risks of perceiving objects in the room which aren’t actually there.
Then a hole tore open in the wall, and tiny robot spiders started crawling out. I walked over to the large gaping hole in the wall, and stuck my head inside the hole in the wall and BOOM, I banged my head right into the wall. There was no real hole in the wall, I was wearing a Microsoft Hololens.
–Anonymous AR Tester
As seen with this Anonymous AR Tester, the ability to affect the user’s visual perceptions is increasingly acute with AR, to the point where they unwittingly struck their head on the wall. In this way, augmented reality effectively projects consciousness directly back into the physical environment, with an added layer of functionality. The latency has become so low as to hack into the lower level limbic system and convince the brain that reality has been visually over-written.
“A factor of four is a wonderful thing in a lot of cases.”
– John Carmack
1D. Brain Computer Interface
Using augmented reality, one could watch a film on an entire wall, host video conferences with holograms, or enact video game quests in real life.
With the brain already directly connected to the body, connecting the body to the computer creates a brain computer interface, using the body as a medium. This engages more of the mind into a data space overlayed onto physical reality.
As abstract AR interfaces progress into maturity, it is possible that formed habits might be well adapted for virtual realities yet mal-adapted to physical realities. This is where post-skeuomorphism retreats from abstraction and returns to skeuomorphism for habit forming UX patterns that are compatible with augmented reality formats.
The goal of this cycle is to provide a human adapted interface for virtual and augmented reality which maintains the human’s consciousness grounded in the physical body.
If a tree falls in the metaverse, will there be any GPU to render it?
Kitbashing is a practice whereby a new scale model is created by taking pieces out of commercial kits. These pieces may be added to a custom project or to another kit. For professional modelmakers, kitbashing is popular to create concept models for detailing movie special effects. – Wikipedia
Hyper-realism and the Uncanny Valley
Over the last several decades, the digital art world learned many ways to steer clear of the Uncanny Valley. Facing the dizzying height of the cliff into the Uncanny Valley, digital artists retreated into hyper stylization as a natural regression into cartoons, pixel art, retro vibes, primitive modelling, and ultra-minimalism to adapt the craft to screens and interactive mediums.
Now with the advent of 3D interfaces, objects are being asked to be redefined as 3D artifacts to become interact-able in the substratum of augmented reality, digital twins and the metaverse.
The artist Beeple is a classic example of skillful kitbashing.
Kitbashing is the process of taking objects from collections of modular geometry and mixing them together to create new objects, characters, and scenes. This is where the millions of 3D models online come together from Turbosquid, Art Station, Sketchfab and 3D Warehouse to form the paintbrush of kitbash models.
As a medium unto itself, kitbashing is an aesthetic of no aesthetic. All polygons are made of the same stuff, cut from the same block, capable of re-assembling themselves into objects of higher complexity and order.
Kitbashing is a technique which defies limitations rather than defines them.
Now with Megascans and Unreal Engine 5, kitbashing tools have crossed the uncanny valley and returned to a hyper real world for experience in VR and a new generation of 3D apps.
This asks the metaverse to not be tied to any specific aesthetics, to embody them all within its substratum, with hyper-realism as the backdrop, foreshadowing augmented reality.
Many creators already contribute to the landscape of symbols and imagery which is ready to import into metaverse spaces. A metaverse is capable of representing digital vaults of GLTF files, including existing asset banks such as:
- IPFS/NFT repositories
- Met Museum
- Smithsonian collection
- Meta Mundo
This is done by the same means of importing a 3D GLTF model into the world and then interpreting it with various forms of metadata. The metadata contains a variety of attributes defined by the artist which can be arbitrarily interpreted by the software creator.
Since not all styles in geometry can be normalized, an emergent style is born, embodied in the style of Kitbashing.
Where suspension of disbelief is required, kitbashing retreats to a linear substratum, such as all objects formed from clay, then painted in a second pass with a common theme of shaders to eliminate draw calls.
–Anonymous Technical Artist
1F. Aesthetics of a Metaverse
Meta – denoting something of a higher or second-order kind.
The aesthetics of a metaverse transcends its components so that it can hold many different aesthetics within its context. This nature, this is what makes it a “meta” verse.
This can be managed with sweeping minimalist transitions between aesthetic styles, such as seen in wide city boulevards, the open sky between buildings, and various districts of the city embodying design protocols. In this way, a city in the metaverse might be made of a thousand villages.
There are several aesthetics which are informed and defined by performance limitations on all platforms:
- Minimalist Shaders
Metaverse assemblies can be optimized at runtime by applying minimalist shaders to all objects when objects appear in the distance, then switching to their full shaders when inspecting the object up close.
- Occlusion Culling
Even with vast expansive views, all of the data in the metaverse cannot be all displayed at once or it would overwhelm the player and their GPU with a tsunami of computations. A robust solution to performance limitations is occlusion culling, which hides objects in the distance, and reveals objects as you come close to them. Occlusion culling will always be a necessary aspect of the metaverse aesthetic, and creative variation can be found in how objects transition in and out of view.
- Foveated Rendering
To save on power and improve frame rate performance, implementations of foveated rendering are recommended for all metaverse applications. This could also be achieved by measuring where the player’s eyes are looking and increasing LOD on objects in focus. Foveated rendering renders a higher level of detail for objects that you are directly looking at, and less details for objects which are on the peripheral view.
Like kitbashing before it, the metaverse aesthetic is practical, non-specific and embodies all other aesthetics, as a medium for mixing artifacts compatible across worlds.
Mature cities contain a variety of architectural styles.
In this way, post-skeuomorphism is a practical integration of skeuomorphism as a self-aware design language, embedded within an abstract world with all of it’s limitations. The limitations are accepted, and suspension of disbelief is boldly generated, broadly swerving away from the uncanny valley while implementing imagery and interpreted functionality from a wide range of visual languages.
A mythological substratum abides in the base layer of human consciousness. In this way, one’s worldview can be overlaid into the digital space imbuing it with that mythic power of storytelling.
–Unknown human sitting by a fire