A.I. in Film-making

The Brutalist, and why we need to think about what we think about Art.

There has been a great deal of controversy this week surrounding the Oscars and, more specifically, The Brutalist; a movie that is currently nominated for 13 awards, the most of any foreign (non-U.S.) film ever. Those nominations are in question now however with many calling for the film's disqualification after people became aware that A.I. was used to assist in the making of the film.

To highlight the scope of what we are discussing here, we are not talking about a film that was "made with A.I." or even one whose script was written by A.I., an issue that has been controversial elsewhere in the film community (see The Last Screenwriter).

Where A.I. was used in this film's production was in post-production, specifically:

- A.I. powered software called "Respeecher" was used to enhance the Hungarian accent of lead actors Adrien Brody and Felicity Jones

- Generative A.I. was used to generate several still images that were used for the film's conclusion.

Laying cards on the table, this controversy is not only an over-reaction to what is, at best, a minor use of technological tools in a film, but is so wildly, ridiculously over-the-top that one does have to wonder if perhaps there might be more behind the backlash. Almost certainly the backlash has more to do with the idea of using A.I. at all than the specifics of what was done in this case.

Regardless, the controversy is there, the calls for disqualifying films that use A.I. are growing louder, so lets take a deep dive into what is actually going on here, what's really at stake, and where things are likely to head; regardless of the noise.

A.I is a Tool

"The computer's a tool! No man, the computer's a toy, you're a tool." - Marc Maron

Partly due to the hype around A.I. at the moment and the billions of dollars being pumped into data centres, power stations and model training; in part to secure a 'win' in the next big technology race and in part, it seems, to satisfy the egos of a bunch of middle-aged men with way more money than anyone should ever have; it would appear that any mention of A.I. results in a very strong reaction. A reaction far beyond what you would expect talking about any other comparable technology. The other component in this maelstrom of commentary seems to be the fact that we're talking about creative acts. Whilst there are definite concerns about software engineers being replaced by A.I., and payroll staff losing jobs due to A.I., there is no general outcry. The discussion there is mostly limited to those affected, those who will benefit, and those professional tech's and commentators looking to fill column inches.

When it comes to generative Art however, or creative text generation, or A.I. in film, there is a far greater hue and cry. This can only be put down to a sort of romanticism around the creative act; A half-forgotten dream from childhood that maybe one day I could be an actor, or an illustrator, or a writer; a dream abandoned but lived out vicariously through the luminaries who made the effort to get there. The Stephen King's, M.C. Escher's and Henry Caville's of the world, the creatives who make a living from creation much to our envy and acclaim.

Of course, like all romanticism, this is a rose-coloured view of creativity as anyone who has worked as one will attest. Being an actor may sound like the most fun job in the world (and for those who enjoy it, it may well be so) but it's also a great deal of gruelling hard work. Much the same can be said of a writer or an artist. During the late 90's and early 2000's when I spent a great deal of time writing in many genres and publications the single most common piece of writing advice I heard handed out was "unless you absolutely have to be a writer because you just can't be happy doing anything else, don't do it.".

It's work, its work like any other, and it is both supported by and affected by technology just like any other industry is. I'm not about to insult the intelligence of any reader by claiming (as some corporations like to) that A.I. will cause no job displacements; it almost certainly will, it almost certainly already has. Jobs will have been lost, or at least changed, by the emergence of this technology. Look back however and realise that the same was true of the art industry when photo cameras were first invented. Do you think the invention of capturing light on film had no effect on the portrait painting industry of the time? It is very instructive to read quotes from the period; absent context you could believe they were Reddit comments from an anti-AI subreddit.

"From today, painting is dead." - Paul Delaroche, French painter, upon seeing a daguerreotype for the first time in 1839.

"If photography is allowed to stand in for art in some of its functions it will soon supplant or corrupt it completely thanks to the natural support it will find in the stupidity of the multitude" - Charles Baudelaire

"This industry, by invading the territories of art, has become art's most mortal enemy" - Charles Baudelaire

Baudelaire, sadly, an enemy of romanticism and a very fine poet, had a lot to say about the evils of photography. Few today would argue that photography isn't itself a fine art but at the time all of the same arguments were made of photography that are made of generative A.I. today. That it cheapened art, mechanised it, robbed it of its inner humanity and replaced it with something that had no spark. Rarely is there so perfect an example of two different inventions provoking an identical response.

For those who would suggest there is a difference between the complexities of photography and the simplicity of A.I. art generation; as someone who has attempted to produce fine art through both methods, I must humbly disagree and suggest that perhaps you have not yourself attempted to produce real art with A.I. tools. A.I. art generation is very much like photography in that, in the hands of a capable workman, it can produce good imagery. In the hands of a master, it could produce great art. In the hands of most users, it is the equivalent of a point and click weekend photographer, and capable of generating little more than the illustrative version of holiday snapshots. Sit down with a conception of a great piece of art you wish to create and try to do it with an A.I. tool and you'll soon realise how much there really is to controlling it, to mastering the process, skills no less complex than the correct handling of exposure, light levels and focus, particularly when done using open weight models and full control of the hyperparameters, weilding prompt and step size much as a photographer might F-Stop and Aperture.

We have digressed however, lets turn back to the question of The Brutalist and dive deeper into what they have done to upset people so. We will focus on the main use they made of A.I., that of perfecting the accents of the lead cast members. This has been called dishonest, "cheating", a way to prioritise performers over language accuracy (though honestly, I’m not sure what would be wrong with doing that in the first place). Many have said they felt disillusioned because they thought the accent was a "personal skill" and were disappointed that it was not.

To address the dishonesty claims first, it should be noted that the reason this controversy sprung up was due to the film's editor discussing it in an interview. The A.I. tool company itself was openly listed in the credits; this is not something the film-makers tried to hide, it likely never occurred to them that they needed to. And why would it occur to them? Lets take a quick walk down some random movie trivia for a moment:

• In 2009, District 9 used a combination of actor performances and digital audio manipulation to achieve the distinctive accents and clicking of the aliens.

• In 2008, the voice of the titular character of the animated feature "WALL-E" was created by sound designer Ben Burtt using a combination of his own voice and electronic processing.

• In 2007 (and later), in the Transformers movie franchies, the voices of Autobots and Decepticons were altered using various audio effects to give them a mechanical, robotic quality.

• In 2001-2003, the Lord of the Rings movies, Andy Serkis' memorable performance as Gollum made use of digital audio processing to add to the distinctive raspy tone. (Though Andy Serkis does give an excellent impression of it even without the effects.)

• In 1977, the voice of Darth Vader, performed by James Earl Jones, was enhanced using a harmonizer to deepen and add resonance to the voice, which created the iconic sound of the character. Quite famously the actor who played Darth Vader in the movie was David Prowse. Lucas never intended to use the on-set recording of his acting performance in the final movie; David Prowse was from the west country in the UK, and sounded more like one of the Lord of the Rings hobbits than the Darth Vader we know today.

Let's not be satisfied with just vocal modifications however, lets go a step further:

• In 2015, Lena Headey declined to act nude for Season 5 of Game of Thrones, and a body double was used in her stead.

• In 2013, in Iron Man 3, a body double was used to film the final scene for Robert Downey Jr. due to a leg injury. The filmmakers added a CGI "mask" of Downey Jr's face to the double, making it almost impercetable.

• In 2012, in the movie Anna Karenina, a body double was used for nude scenes in place of Keira Knightley, whose tattoo's were seen not to match the character's demure nature.

• In 1991, Terminator 2, Judgement Day, Linda Hamilton's twin sister stood in for her at multiple points throughout the film.

By now, hopefully, we're starting to reach a similar conclusion. The film's use of A.I. to modify a voice in order to achieve an effect they wanted, far from being a new and unusual use of a disruptive technology, was simply a new way of doing what film-makers have always done - namely, whatever is necessary to achieve their vision for the art they are creating. No wonder David Jancso, editor for the Brutalist, thought nothing of talking about the A.I. tools he used in post-production - he had likely been using similar tools to do similar things in films for his entire career.

Most of those were semi-modern examples, lets try one more. In 1952 one of my favourite musicals, Singing in the Rain was released. It was set even earlier in Hollywood at the dawn of the "talkie" era, when recorded spoken dialogue first began being used in films. Famously this was a difficult transition time (much like the one we're in now) as many actors from silent movies who had been quite popular for their physical acting and appearance were incapable of bringing the same artistry to voice, making them unable to transition correctly. (Much how, in a reversal, many radio drama performers were unable to make the transition to TV dramas as in-home television slowly overtook and replace radio sets, giving rise to rather unkind saying "he has a face for radio.").

In the movie, Singing in the Rain, this exact scenario is used as a key part of the plot, with a young wanna-be dancer and actress (played by Debbie Reynolds) is hired to re-dub the vocals of an established actress (played by Jean Hagen), as the established actress' voice was not considered suitable for her on-screen image. In a beautifully ironic piece of Hollywood trivia, the producers of Singing in the Rain preferred Jean Hagen's natural speaking voice to Debbie Reynolds' Midwestern accent, and so her voice was itself replaced during the dubbing scenes with the real voice of Jean Hagen.

Anyone claiming the use of A.I. in film-making is somehow "cheapening" the art or damaging it's "authenticity" has missed the point entirely. Hollywood is called the dream factory for a reason - this is film-making, and this is acting. It is a group of people working together, with the technology they have available, to make the unreal real. They are playing pretend on a grand scale for our amusement. Anyone who finds themselves dismissing the incredible amount of hard work, vision and artistry of the dozens of people who came together to create this piece of art simply because two voices in the movie were altered by a computer tool, in the year 2025, is not only working extra hard looking for a reason to be upset; but is also dismissing and discarding the work of far more creatives than could have ever have been affected by the use of this tool in the first place.

Let's hope no-one tells them that Mr. Ed. didn't really talk.

The Conversation to Come

I find many of these controversies particularly tiresome as they are too often all emotion and outrage and precious little logic. There is no material difference between using an A.I. tool to modify an accent rather than a non-AI tool to add a different sort of effect. You could go further and argue there's little difference between using technology to make this change and someone using their vocal chords to change their accent when they aren't really Hungarian, in the end you're seeing something on the screen that isn't real; but that's what film and TV, that's what Art in general, is for. No amount of belief in the quality of his performance is going to make Hugh Laurie an American doctor called House; he has quite a nice British accent naturally and while I expect he would be an extremely entertaining dinner guest, I wouldn’t want to approach him for a medical diagnosis. None of this, none of the fake accents, custom built sets or scripted dialogue detracts from the pure emotional power of his performance in that show.

Despite my firm belief in this though, it is evident that a broader discussion definitely needs to happen within the television & film industry. Whether a voice of reason prevails and this is seen as an extension of the special effects we have always used, or reactionary opinion holds the hour and we see a wide-spread pushback against A.I., the key thing is that we can't relegate this to the shadows or off-the-cuff last minute decisions pressured by a particularly vocal subset of members of the community (and industry). The Academy, for one, is going to have to make a call here and likely a hasty one, but a longer and more nuanced discussion is really due this topic. This technology exists now so if it is to be singled out for some sort of regulation or exclusion then the Academy needs to make it clear going forward what their policy will be; allowing film-makers the opportunity to take their opinion into account.

There will likely be calls for explicit disclosure when A.I. is used in film. Again, this seems counterintuitive as such a requirement has never before been placed on the wide range of editing and special effects tools we use, but if this is something that people want then the discussion needs to happen, preferably without counterproductive hyperbole and doomsaying. We're likely overdue a more broader discussion on Art and film-making in general as well as the "authenticity" and "unfairness" arguments being passed around are really illustrating how far the general romanticised view of filmmaking has drifted from the realities of what the industry does.

The Positive Impacts

Lets not end this bleakly however, thinking only about the potential for change, job displacement and altered creative environments. Let's finish out with a few possible future benefits that this technology could have for people working in the industry.

A.I. has the potential to improve efficiency, to make things we already do faster and easier. This is far and away the area where A.I. has had the most success so far, acting as a tool and assistant to a creative rather than a replacement. A.I. can modify voices and images but it can't yet plan and organise with the creative vision of a great film-maker by itself, and may never be able to. With efficiency comes reduced costs, and with reduced costs for making movies comes the possibility not only of bigger, grander projects, but of making more movies and taking more creative risks. This, paradoxically, leads to a higher need for skilled creatives rather than displacing them.

For voice actors this technology has a lot of promise. Not only does it potentially help them work faster, and thus be able to take on more work if they wish, but the ability to make use of these tools provides a talented, expressive voice actor with opportunities beyond those which she might be able to obtain were she limited to her natural vocal range.

Far from the claims made of A.I. harming authenticity, this technology also stands to help a film-maker get even closer to their vision for their film then ever before. There are many who much prefer to watch a subtitled film than a dubbed one for this reason; not only are dubs sometimes quite notoriously bad; so as to be almost unwatchable; but there are many movies where watching in the native language, the way the characters were meant to sound, will unlock an entire new layer of the film to its audience. With these technologies, in theory, a filmmaker could dub films into other languages with the character still speaking in the original actors voice, with an appropriate accent and language for the targeted audience. This could stand to greatly increase accessibility of movies and TV shows to foreign audiences.

Finally, on the subject of accessibility, lets consider the effect that such tools have on the accessibility of Art in general for creators. In my teen years I was lucky enough to have access to a camcorder, a rarity in those days. With this in hand I recruited a cast from school for a weekend to create a very silly short film as a school project. With a camcorder, a tripod, and some willing actors, we put something very primitive but amusing (to us and our peers at least) and had a fun, creative time doing so. Editing was primitive and involved multiple VHS players and multiple blank tapes as cuts were made back and forth. There were no special effects other than what we could build ourselves and the sets were a backyard and a living room.

Despite the primitiveness of the whole thing, we weren't that far removed, years perhaps, from a time when this project would have been completely impossible for a small group of school yard teenagers. Editing once required specialised expensive machinery (and a pair of scissors), special effects meant hand-painting celluloid, and even the simple filming we were able to do with the equipment available to us would have required a small film crew instead of one young man acting as director, cameraman and editor.

Now? There are free digital editing suites that are powerful enough that they are used in Hollywood itself. Just about every teenager walking the street has a high quality camera in their pockets capable of filming in resolutions that professional cameras would have dreamed of back then, let alone my little camcorder. Any teenager could start such a project on a whim, and the audience for the finished product isn't limited to a classroom of fellow students watching on a borrowed VHS player; digital distribution makes it cheap (free) and easy for their work to be distributed to potentially millions of viewers all over the world. That's what can be done with today's technology.

From 2018-2020, a single 3d artist named Syama Pedersen released five episodes of a short-film called "Astartes", a fan work based on a well-known property owned by Games Workshop. This work was made using easily available and well known technology and was so well received that Pedersen was later hired to make an official sequel to his work by Games Workshop themselves. Already the technology available to us has allowed creatives with a vision to produce things that would have been impossible 20, 10 or even 5 years ago.

With the advent of this new technology, what will be possible for the aspiring creatives of tomorrow? Rather than fearing A.I. and its effect on creative industries, I can't wait to see what happens next.

-Nick Bronson