Research published

ZKM, the german media art museum, published my research on Strategies of Arrival, done under guidance of Bruno Latour.

It’s made up of three parts:

Inhabiting a map

How to enter a space

and

Instruments of reation

The first one is the most spectacular I guess, but make sure to check out the others too. And, once the museums reopen, check out my piece at the current ZKM exhibition Critical Zones.

AI deleting human bodies

I used neural networks to recognize and remove human bodies from videos, simultaneously trying to fill the void they left. Why? Well, as we are externalizing not just our bodily actions to tools, but our decision process to all-knowing, all-loving AIs, the traditional, perceivable image of what constitutes a human body has become obsolete. It seems oddly fitting in that case, for disembodied neural networks to remove our outdated bodily image.

Mira Hirtz performed for the camera. I then used Mask_RCNN to do body recognition and Generative Image Inpainting with Contextual Attention to fill in the emptiness.

This was a test for Bruno Latour’s upcoming exhibition on the Critical Zone. Made in November 2018.

Zuckerberg emojis
Adding human body to digital expression

Squint your eyes!

You’d think an emoji would be a clear and precise representation of an emotion. However the difference in rendering on different devices provides a great potential for miscommunication.

But even on the same platform, the kind of emotional reaction you may express is curated through the choice and design of available emojis. Take Facebook for example. The five Facebook emojis give you broad expressive capabilities, yet their design is very specific - and close to the extreme ends of the emotional scale. When using emojis, we wear these slightly too big masks of these particular, predefined facial expressions. We are becoming them.

I decided to reverse the process, to add body to the digital expression. That body should be, of course, that of Mark Zuckerberg.

Here are the results, created through disembodied neural networks.



I think CNNMRF grew the eyes from the hair, from the imagined eyebrows it found.

How it’s done:

First CNNMRF populated the emoticons with features of Marks face. It especially likes eyes and hair. I then used Deep-Image-Analogy to add more features, contrast, structure and adjust histogram. Finally jcjohnsons neural-style was used to upscale the image while adding texture.

1.Original 2.CNNMRF result 3. Deep Image Analogy output 4.Upscaled with Neural-style

Stickers for Telegram now available: https://t.me/addstickers/cursedmark.

Got a question? Just ask me.

Neural nets see cancer zombies

Looking for ways to visualize machine learning processes. Neural nets are trained to recognized cancerous Ki-67 marked cells in biopsies. Instead of just counting the cells, a neural style process is run to produce infinite zombies in their place.

Visualizing complexity


I was doing some research for an upcoming exhibition and looked into the question of tone in scientific communication, specifically the stylistic influence of visual representations of knowledge. Obviously the style changed over time, became more abstract, less naturalistic, less.. certain. As if the scientists became troubled by the idea of adding some character to the visual language - which is futile anyway: no matter what you do you end up with something that has character, just maybe a lame one. One particularly popular type of illustration in geosciences is the block diagram. While visually appealing, it often lacks any kind of a sensible way to embed processes that happen outside of the cut out piece of earth - the circulation of water, the circulation of life forms, human influence, changes over time - all that has to be added in some stupid way, with arrows and explanations.

Instead, I looked for ways to bring some of the old, more subjective ways to scientific illustrations. Add a few too many details. Make it alive, permeable for living creatures.

I took some illustrations by the 19th century German zoologist, philosopher and illustrator Ernst Haeckel, and used them as style for a neural style transfer algorithm.


We are conditioned to look for meaning in scientific illustrations. While the meaning is lost through the process of neural style transfer, the illustration gains a lot of visual complexity. As viewers we may sense a deep cross-interlocking between the animal and its environment, but we do not get the chance to actually understand the specifics of their relationships.

Here are a few more:








Because of the non-deterministic nature of neural style transfer, the results differ a little bit every time you run the script. Combining a dozen outputs creates an interesting effect. It’s not breathing nor movement but some kind of uncertainty that normally only comes when you freeze living matter in time. Like a collection of portrait photographs, neither of which is truly you.


How to do it yourself