Matt Swoboda

In this episode our dear friend Elburz Sarkhobi talked to Matt Swoboda, a developer who started his working life in video games and as a principal engineer in Sony’s R&D team where he developed graphics techniques on PlayStation 3/VITA/PlayStation 4 that were used in over 200 released titles and is now the director of Notch. 

Can you tell us a little bit about your background? I believe you started in the gaming industry and even in the Demoscene, how did you end up where you are now?

So... I was in the Demoscene in the mid 90s, I guess, since I was a teenager actually, and I don’t know if anyone has ever heard of it apart from a few of you, but the Demoscene was kind of the original creative coders. So people who, way back in the 80s and 90s, were using computers and real time graphics to make art and make creative, often linear and non-interactive narrative projects. Of course, the world is full of this nowadays and there are huge companies made up of that. So I go into real time graphics thanks to that. It was super fascinating to me that you could make something so quick. I remember back then we were trying to use POV-Ray: this old ray-tracing program that would take overnight just to render some shiny balls. And with real time graphics, you can do it at the snap of a finger. You can just make something and it is there for you to change and modify. And it was kind of addictive...that rate of work is addictive. And I got into video games because back then if you were into graphics and real-time graphics, there wasn't really anything else available. So I got into games, I made some games for PlayStation II, which was really fun. I joined Sony Computer Entertainment eventually and was in the R&D team for various of the PlayStation consoles doing pre-launch, working on software and graphics-based platforms. The only thing I ever did that you might have seen is the background wave when you turn on the PlayStation Four.

Oh, I know that wave. Thank you for that wave.

I imagine they’ve probably changed it by now. So for a while it was pretty cool, then I realized that even though I was working with video games and I liked graphics, I didn't really like video games anymore. So, early in 2010, I needed to get out of there and do something else. I was doing some sideline stuff, installations and bits and pieces, and I thought maybe there's a career here. And so along the way I started to make demos for Demoscene and building really sophisticated tools just to do stupid hobby projects. And at some point I thought maybe there's something here, maybe there's room in the world to make a tool that's a bit like what we have for making demos, but to make graphics in real time.

It didn’t have to be about making real time stuff that runs in real time, but using the workflow and everything we’ve learned over the past like 20 something years of doing this; maybe there's a way to package that up, put it in a box and then give it to artists and see what happens. And so various lucky coincidences happened. I met Ash Nehru and decided to collaborate with him and the first show we did was for Eurovision in 2014, it was the first big show I ever did. And there were like one hundred fifty million people watching it on TV.

That's really big for a show!

Yes, we were thinking If this doesn't work, we need to get out of here real fast. We’ll get the hell out. There's a lot of people who are going to be angry with us...but it actually worked out really well. The second show we did was a music video for Coldplay and then we did the Ed Sheeran World Tour. And that really kicked it off because it snowballed and artists started asking for the same things we did with Ed Sheeran. Like Sharon Osbourne called someone and tried to figure out who was making it, and we ended up on the Ozzy Tour and then on the Coldplay tour, and various other ones and suddenly it actually became a thing, it started gaining traction and then we just kind of rolled from there. We hadn't even started a proper company. It was just me and John, the director, and then we got Luke to join us as well. We became the three founders of the company, and we actually started a company and made a website and did all those serious things you have to do when you're a grown up and have a company. We always kind of wondered, is there any point to this? Does anyone even want it? And it was kind of interesting when it turned out they did. It was actually great.

Do you find there's still a bit of that Demoscene attitude or approach when you guys work on Notch now?

I find that the Demoscene people are often a really good fit because you get these people and developers who have a really wide range of experience and a creative edge, as well as a technical edge. To be a developer in this industry, in this world, you have to be a bit creative as well. It's not like we're making databases here. We like making tools for artists. So you must have some knowledge of why you're doing things. One thing I learned from the games industry was that there were a lot of really terrible tools made in the games industry and all the games companies had their own tools. It's changed now with Unity and Unreal, most companies seem to be using those instead, but back in the mid 2000s, every company had their own tool and they were all horrible. And the thing is that when your company develops a tool then you are forced to use that tool, you don’t have any other choice and that’s that, but when you are actually making software and selling it, you have to really work on making it good because no one is going to use it unless they want to. 

I think that's probably a good transition to talk about what Notch is for folks that are maybe unfamiliar with Notch.  What were some of the initiatives or business plans or ideas that you guys had before Corona came around?

There are a few things we did. On the business side, we wanted to really highlight our interactive capabilities and the fact that it is super easy to make an interactive piece in Notch as well. I could literally plug in a sensor or two and manipulate that live so you can easily make something interactive. We really wanted to highlight that so we’re doing a ton of work on campaigns to show people how effective and easy it is and also just what can be achieved. Because I think in our line of work you really have to show the possibilities. And we are also heavily working on training and making it increasingly accessible. 

We also always wanted to balance the capabilities for people executing projects, but also focus on workflow because the whole point of this tool when you are working in real time, is that rendering is fast. That's important. But then everything else becomes really important as well. There is no use in having a super fast render, if it takes you forever to click everything together, you might as well just do it in a different software with slower render but that is quick to make. So you have to keep working on the workflow all the time. You want to make it feel as creative and intuitive as you possibly can. So we are really trying to focus on that.

That sounds like an interesting balance to try and maintain what is working fast versus rendering fast; it almost seems like at times they might compete against each other, you know in terms of whether it's in the back end or in the front end.

What I think is unique about Notch is that besides the possibility to work on real time, there are other real time graphics engines, but Notch is there to function as a creative tool. We're using a lot of techniques that a lot of other real time engines could be using as well. We have some of our own, but a lot of this stuff is shared. But what we are trying to do is package it in a way that makes it feel intuitive. 

And I think that's one of the most interesting features.  That sounds like something most creators really dream about because there are different levels of creators. There's the folks who want to click a few buttons and get a great result on the output. Don't really need customized ability.

Even node systems are not the same, but Notch is not just a node-based system, you use a lot less nodes and it feels more layer-based than node-based. We use node ordering to control the effects, just because I think it’s intuitive. So you don't end up with these huge, like, rabbit hole projects quite as much, where you have like ten thousand nodes and you're zooming in forever and ever and ever.

That's the story of my life right there. So... before Covid, there was this push towards balancing creativity and technical aspects, rendering speeds, going more towards installation artists and letting them know this is the tool you have to use. And then, all of a sudden, the industry shuts down. We had some kind of industry D-day almost, where all the concerts, all of the gigs and shows you had, are gone. How did this affect your plans or maybe originate some new plans for Notch?

Nothing had ever been shut down this fast. How could you predict that the whole industry would stop operating in 15 minutes? Because with the touring industry, we saw it coming first because shows were getting canceled and concerts got canceled before the lockdown started really happening. The first thing we thought was that this was really going to affect our customers, our users. This is really going to hurt people, so, what can we do to work with them to try and help out, to keep them going? And so we did a few things straight away, we did some community initiatives like another Notchmeister competition because we thought, people are home, this is something they do, give them something to focus on and enjoy. We really quickly worked on getting a learning addition and really dropped the price because we wanted people to have the opportunity to learn. And we did some things for our existing subscribers as well. Look, we're all in this together, right? Because we are a step removed from the front line. We're still working. We just moved. We all shut down the office, went home and then we took it from there. But we're one step removed from the people whose shows all evaporated in two days. We're one step removed from that, so still very closely related to them. If they can't work and they can't use Notch, then that really affects us, too. So we want to be in it together.

And it sounds like a lot of these endeavors are really focused around community in the sense of Notchmeister, which if folks don't know about, it's kind of a content creation competition that Notch holds with some interesting criteria, you create some beautiful art and you submit it and there's a chance to get some recognition and win some stuff out of that. Was your focus immediately shifted towards supporting the community?

I think we indeed really quickly focused on our community and what we could do to help our community. It was very important. But as far as technical things go, we quickly realized the livestream was going to be a thing. So we quickly added some features to the product; for example, I'm running my camera through Notch now, so we made a plug in so that Notch can emulate a webcam so that you can go to your zoom call and have the ability to run stuff. So you can use Notch straight into Zoom or Hangouts. Maybe this is a really great way to cheat at pop quizzes and use a glitch effect if you want to stop talking to someone. We also added the ability to go straight to Facebook Live, YouTube, Twitch straight from Notch. 

Do you think that is the biggest potential new kind of business or market? Are some of these things going to become new staples of your business, like supporting live streams? Are there other things that you've seen crop up that you hadn’t thought before, but that you actually think is a really good thing that you can start doing.

Well, the interesting thing that's come up is XR/AR. After the lockdown eased a bit  we got into a situation where a pop star or an artist could go into a studio and record, but they just can't perform live in the same way. So it opens it up enough that the natural evolution when we’re at home is XR/AR shoots, so you are either using a green screen or using a LED screen, tracked cameras and making the star able to work in a virtual environment or in a virtual world of content. A good example is the American Idol final where the Katy Perry performance was all an XR shoot so you had tracking cameras and LED, render in real time and it was heavily using Notch. The content of that piece was made in Notch. That’s a really nice example of what can be done, because it's a live creative performance. I think this technology has been coming up for a while, but I think it was a technology that really lacked the application. And suddenly there's the need to be delivered, so suddenly it makes a lot of sense, a lot more sense than it did six months ago. So we've been doing stuff to support that as well. We have a new green screen here...everyone's got green screens now, apparently green screens are really helpful right now, they are sold out. 

Do you think virtual production such as XR/AR is going to become like a new standard?

I think it's already happening. There are limited options for people to deliver a show right now. And it's interesting with live events. Someone told me that live events is one of the most robust, reliable industries in the world. So during the Great Depression, and all kinds of recessions and global problems that have happened in the last hundred or so years, live events always bounce back. They come straight back because people want to go out. They want to. So it will come back. But people still want to have those experiences, even if they can't go outside to do them. I mean, all the things that we were doing before in terms of shows and performances, they existed because people wanted them. Human beings need to go and spend time with each other and go to parties and go to shows, but I think it's going to take a while.There's a whole issue at the moment because artists can't get insurance for COVID, they can't travel because no one will insure anybody for it. So that eliminates the ability for touring artists to travel unless someone else picks up that bill. If someone gets sick while touring, someone is going to have to pick up the bill for that. It just doesn't work. So there are practical limitations, I think what's more interesting is that the world will be different by the time things bounce back, for example, we’ve all had a year of doing different things in a really simple leve, for instance, everyone's going off to learn new software or learn new processes because they've had time to do it. So things will be inevitably different.  We just have to make sure that we can do the best to be part of things when they come back and still be relevant and interesting. Continue with the things we were doing before, like working workflow and usability and UI, that there's still absolutely the right things to be doing, Covid or not. But then small shifts, shifts in focus to support what's going on right now and hopefully support people. So when they are working, they are able to work effectively with Notch. That's the right thing to do.

So, from that perspective, I can almost see our industry coming back stronger because essentially we're going to have this renewed talent pool where everyone had some time to finally decompress a little bit from just back to back crazy projects, and had a little bit of chance to expand their skill set. Do you also imagine a scenario where the “new normal” is actually going to be a stronger industry than what it used to be before Covid?

Yeah, I hope this balances things out and makes sure the people get the right recognition for their work in the future. Maybe some practices that were the most healthy remain. I think there will be new things, and it’s not all positive, there are going to be effects. There are companies that are not going to come back from this, there are companies out there who have got a lot of trouble. And I think we will see like there will be companies that go down. 

Are there some things that Notch is thinking about in the long term going forward?

Well one of the things that this is bringing in is the fact that the content is traditionally made in the studio and then the creatives never go on site. And it’s important because when you're making a physical piece that has to exist in the real world you need to actually be able to see that in the real world as it is going to be used and be able to change it in order to get a better result than if you're just guessing based on what you see on the monitor. So it's brought more creatives into the room because real time rendering has kind of almost forced them to come down and be on site making the changes rather than just being able to send video in from remote. The best thing you can possibly do is get more creatives and get the creatives to be in the room and get them to be the ones able to do the work. You don't want the technical powers to be so high that the creatives can't be involved. You don't want them to have to go through a translation of someone else. So I think that's a really good thing that can be done.