The RetroBeat: The Video Game History Foundation is on the hunt for source code

The video game industry is still young, but we’re already in danger of losing parts of its history forever. That’s why the Video Game Source Project by the Video Game History Foundation is so important.

This project is a call-to-arms for the industry to locate and preserve source code, the original art and programming that make up our beloved games. For years, keeping this source code organized was not a priority for many developers and publishers. This can make it difficult to preserve the original versions of these games. It can also hinder historians who could learn about a game’s development by looking through those files.

To show just how interesting and important source code is, the Video Game History Foundation is hosting a digital event today with Monkey Island creator Ron Gilbert. A $10 ticket gives you access to the live show and a recording of the event, which will pour over The Secret of Monkey Island‘s source code — including content cut from the final game. The stream starts at 1 p.m. Pacific today, but since you get a recording of the show, you’re not on a deadline.

I talked with the Video Game History Foundation’s co-directors, Frank Cifaldi and Kelsey Lewin, about this source code initiative and its exploration of The Secret of Monkey Island.

Go to the source

GamesBeat: Why did you start this Video Game Source Project?

Frank Cifaldi: We believe that there is no better way to study how a game was made than access to its source material. When we say source material, we mean anything that was used in the production of the game. That could mean source code, and it often does, but it also means things like original art that was produced, original sound and music, documentation, correspondence, just anything that survived that gives us more direct, behind the scenes access to the game. That’s going to give us so much more than the game itself could.  And at the same time, this is the kind of material that is very unlikely to be accessible to people. It’s something that’s held as a trade secret among companies. It’s something that, until recent years, was not saved and catalogued to our satisfaction. We’ve already lost a lot of these source materials.

With the Video Game Source Project, it’s twofold. It’s an attempt to call out, to the industry and to researchers, the importance of this material to telling stories. It’s a callout, but it’s also a call to arms and a demonstration.

Kelsey Lewin: And it all goes back to our core mission, which is just that we want to see more video game history in the world. We want to see more books, more documentaries, more research being done. We don’t have access to a lot of the stuff that makes that research, writing those books and making those documentaries, easy. Or even doable in a lot of cases. This is a thing that we think would help bring more interesting stories to light.

Cifaldi: This is not unheard-of in other industries. This is just how history books are written, with access to material. You don’t write a biography about George Washington unless you read his archived letters. You don’t write books about how a film was made unless you have access to things like its script and maybe even storyboards and any other behind the scenes materials. It’s just not very common that those materials exist in an accessible space for video games. We haven’t solved that yet. It could be years away. But the Video Game Source Project is what we consider the first step toward getting to this future that we want to see, where it’s normal to be able to access this stuff instead of sort of taboo or perhaps stolen from Nintendo’s servers.

GamesBeat: Where are you most likely to come across source code?

Cifaldi: From the people who wrote it. We have maybe about 100 repositories right now in our archives that were almost entirely sent to us by people who were involved in the development of the game in one way or another, be it single author, or they were part of a team. That tends to be the only place that the source exists, especially for these older games. It’s from people who took it home. We don’t believe that many video game companies have maintained archives that extensively go back to the ’80s and ’90s. Especially for games of that vintage, most of them likely only survive in what we call private collections. And so part of the point of going out with this big splash that we’re doing with the source project is just to have this awareness campaign for people who have material like this. Hey, maybe this stuff should be somewhere. Maybe you’ve been holding on to it, waiting for it to make sense to donate it somewhere or give it to someone. We should start that conversation now. We’ve added quite a few things to our collection because of that.

GamesBeat: Why was archiving such a low priority for these companies in the ’80s and ’90s?

Lewin: There wasn’t really a secondary market yet. We’re in an age now where there are lots of remasters and ports and that sort of thing, but back in the ’80s, if you made an NES game, you put it out on the NES, and you were done. Move on to the next project. There wasn’t an economic reason for companies to do this. They didn’t have a reason that would make them money, to save any of this stuff.

Cifaldi: That might sound, I don’t know, heartless, but that’s how companies operated. Unless there’s a financial reason to do something, they’re not going to do it. Even if they maybe did archive some of this material back in the day, a lot of it probably wasn’t maintained. We know of some companies that are still around that did archive things in the early ’90s, but it’s kind of stuck on an obscure tape format, and the person who backed it up doesn’t work there anymore, and no one knows what software they used to back it up. It’s a solvable issue, but for a company, unless they have a commercial product or some other commercial reason to go and solve those issues and read that data, they’re not going to do it. It really just comes down to, when we’re talking about things that are lost, it’s money, but it’s also the nature of the industry.

There have been hundreds of video game companies that have gone out of business since the industry’s inception. The rights to those games may now have reverted to someone else, but it’s pretty unlikely in most cases that code survived and was transferred. Or things like, we know of source code being destroyed in office moves. We’re moving offices, and we don’t have as much storage space now, and there’s this closet full of old stuff that we’re just going to toss, because we don’t know what it is. There are so many reasons stuff gets lost. It’s not unique to video games. We see similar stories throughout film preservation as well. It’s just that we didn’t think there was a good, organized effort to call all this out and start talking about these problems before now, so the source project is step one for us toward fixing all this.

Above: These floppies contain the source files for an NES game.

Image Credit: Video Game History Foundation

GamesBeat: What kind of a timeline are we under? How long can old media like floppy disks hold out?

Lewin: We’ve lost stuff already.

Cifaldi: For sure. Nothing that was going to break your heart, but we’ve had floppies here that are pretty much toast. It depends on the media. We can go down a whole rabbit hole here. I’m a bit worried about magnetic media from the 80s, things like floppy discs. This might sound counterintuitive, but I’m extremely worried about optical backups from the early 2000s. CD-Rs and DVD-Rs. By the time you get to the early 2000s, those things are now mass-produced consumer products, and we’ve found that the cheap spindles of 100 discs that we used to get in 2003 were cheap, not just to us but also in the manufacturing process. I mean, it’s not a significant percentage, but we’re starting to see discs of that vintage delaminate, which there’s really no recovering from. We’re pretty worried about that.

But even if, physically, data survives on some formats, there’s also a danger of just knowledge loss, of how to recover these things. We’ve managed to recover data from obscure formats, from DAT tape backup and stuff like that. But it’s only been through having a network of smart people who are interested in this stuff that we’ve been able to do it. I’m worried about losing that level of expertise, even, when it comes to this stuff, even if it’s on a format that could survive. It becomes more and more specialized to recover it. It gets worse as time goes on, to be able to seek out those experts and recover these as those experts age out.

Lewin: What was that compiling program that you found at a company that still owned it, and nobody in the company was able to figure out how to get you this program?

Cifaldi: I don’t want to specify who it was, because we did end up finding a copy through other means. But there’s one game where we had the source. We had everything but the compiler, the actual program you would run to compile to source code into an executable binary for the target system. We had the raw assets, but we couldn’t make the game to play with them. We knew exactly which version number. We had the batch scripts for building the game. The batch script says, run whatever.exe. We didn’t have that .exe. We found the person who wrote that compiler in 1994 or something, and he said, no, I didn’t keep any of that stuff, but here’s the company that owns all of it. I contacted them and talked to their customer support first. They said, oh yeah, we have all of these, we have to maintain all of those, but you have to go through sales to get this compiler. Then I spent about a week back and forth with sales trying to buy a version of their software from 16 years ago. And there’s just not a path for that. There’s not anything inside of that company that will do that. There just did not seem to be a way to get that old piece of software to build this game through the company. And so we kind of gave up on that, and obtained it through other means. Which was its own strange story. But I don’t want to get into exactly how it was obtained.

GamesBeat: It sounds like you have a lot of adventures.

Cifaldi: Yeah, adventures through deceased developers’ old hard drives. Adventures through shady Chinese piracy sites, just to get these things running again. That’s another thing that degrades as time goes on. That compiler I just talked about, the place we found it was a really weird corner of the internet. And it was the only place we could find it. If that weird corner of the internet goes away, does that program go away forever, and is this game forever unbuildable? It’s actually pretty scary.

Above: The Secret of Monkey Island.

Image Credit: LucasArts

Return to Monkey Island

GamesBeat: Along with announcing this initiative, you’re starting with this big Secret of Monkey Island showcase. Can you talk about how that collaboration came to be?

Cifaldi: We were donated a repository that, among other things, had what seemed to be the complete buildable source for The Secret of Monkey Island and Monkey Island 2: LeChuck’s Revenge. I’m a fan of that game, and a historian, so for me this is like, cool, well, the rest of the VGHF shuts down for a week while all I do is go through this. You remember. It was like 12 straight days.

Lewin: Yeah, I decided to just not bother you for a couple of weeks. I’ll just do my own thing over here.

Cifaldi: In looking at all the code and figuring out how to build the game and working with the old Lucasfilm tools and things like that, I managed to learn a reasonable amount of SCUMM, the scripting language. I’m like high school freshman-fluent in SCUMM at this point. I can read it. I don’t know if I can speak it or write it, but I can read it, and mostly understand what I’m seeing. I learned just enough to start doing things like finding areas in the game that aren’t used, that aren’t even on the disc or anything when you buy it, because it wasn’t compiled, and restore them back to their original functionality. I was able to work with our team here to reverse engineer some of their graphic formats, figure out how they work, start spitting out all the frames of information into GIFs and things like that, and find even more older, weirder content.

Essentially, I was in an archaeological dig in what might be my favorite game. I was discovering all kinds of things. I was answering all kinds of mysteries that have been around with this game. Not the secret of Monkey Island. It’s not revealed in the source. I’m sorry. But it confirmed a lot of things about the development’s history that I doubt even the people who made it remember. I was discovering a lot, and it just seemed like something that we could do something interesting with.

We contacted [Monkey Island creator] Ron Gilbert, first of all, just to make sure that we weren’t doing anything that might upset or embarrass him. We’re kind of digging through his garbage, you know what I mean? Looking for things he threw away. We talked to him, and then we also talked to Lucasfilm, because we didn’t want to hit them over the head with it, that we were publishing content around this game. We’re sort of digging through source that was donated to us. And so Lucasfilm was very receptive to what we were doing. Let’s face it, Lucasfilm, they’re the Star Wars company. They understand that fans like that behind the scenes stuff. They understand that it’s good for the franchise if fans can talk about it in a more substantial and interesting way. I think they understood. They applied that feeling to Monkey Island and gave us a semi-official blessing to go ahead and publish content. And then Ron is a very transparent person with his development history. He had no opposition to anything we were showing. Nothing in there is strange or embarrassing, I don’t think.

We thought, since the anniversary was coming up — we’re in it right now. October 1990 is when the game came out. We don’t have an actual date. I don’t know where the October 15 date that’s floating around came from. I can find no source for that. But we know it’s October. We asked Ron, would you be willing to do a livestream with us, a ticketed livestream as a fundraiser celebrating the history of this game and looking at this behind the scenes content? And he was happy to, so that’s what we’re doing. We expected to sell maybe 100 tickets. We’re at 985 right now [as of October 27]. I don’t know if surprised is the right word. Delighted is in there for sure. People obviously are interested in talking about how games are made on a deeper level. The fact that we were able to sell 985 $10 tickets and counting just goes to demonstrate to us that we’re on the right track. We’re doing the kind of work that is going to open new doors and help us talk about video game history in more interesting ways. Which, by the way, is part of our motivation too. It’s sort of the unspoken part. Kelsey and I kind of get bored by traditional video game history narratives. Is that fair?

Lewin: Yeah, I think that’s fair. We hear, because there are kind of a finite amount of people who historians have had traditional access to, interviews and being very open with their work, what’s popular, all of those things together, we end up hearing a lot of the same stories over and over again. It’s good that these stories are being told. It’s not that we shouldn’t be telling the story of Pac-Man or ET or whatever. But there’s a lot of video games, and a lot of stories to tell.

Cifaldi: It’s not like Monkey Island is uncharted territory or anything. But looking at that game through its artifacts that were left behind in the development process is something no one’s ever done before. Well, really the reason that we’re creating content as part of the source project is that we want to inspire people to think about and investigate video game history a little differently, to start going closer to the source and being archaeologists in that way. It wasn’t a big leap of logic for us, that this is what historians would want to study. Sure, you can look at other mediums and compare. But also, when we tend to talk about development history and what people study to look at that, we see two things. People are really fascinated by published screenshots and video of a game before it was done. People obsess over minor details of things like Mario 64’s old HUD graphics and the placeholder audio they had. People obsess over those small details from earlier visions.

They also tend to obsess over what’s still left in the final game that isn’t used. Like the cutting room floor wiki is an extremely popular website, and all they do is go through shipped games and data mine them and find things to help paint that development history a little more. For us, well, what if you could get rid of those abstraction layers completely and access those files and see what’s going on in places that aren’t in the game anywhere? We hope that this is the start of normalizing this. We hope that authors will start throwing their old code on Github or the Internet Archive, just get things out there so that people can start using that as an educational resource, and start understanding development history on a level they haven’t before.

GamesBeat: What was one of the cool things you found digging through Monkey Island that you can share with us?

Cifaldi: Maybe not the coolest, but a couple things that come to mind. We already teased a room in the game that isn’t in the final product. That one was particularly cool because it’s fully fleshed out. It seems finished. It has a really great piece of animation with this severed leg dripping blood. What’s fun about it is that there’s this news report from 1990 that’s been on YouTube for a few years, where they visited Lucasfilm Games, and they actually filmed Ron Gilbert showing off the Secret of Monkey Island while it was still in development, and the one place they show in the game is this room no one had seen before. People were like, what’s this room? Where is this? Someone asked Ron, and he didn’t even remember it. He had no idea. Things got cut all the time.

Above: The cut room!

Image Credit: Video Game History Foundation

It’s one of the first things I look for, because I knew it was cut content, and I found it. It’s in there. This room is not important to the game. It’s not this huge grand idea. It’s a room connecting two rooms, and in the final game they just connect to each other. This room separates them. That in itself is something that is worth talking about.

I think that a lot of the discussion around cut content in games tends to maybe amplify this level of mystique that was never there. Because game development is so secretive, because we don’t tend to get behind the scenes access to game development, we tend to think of it as being sort of mythical, when in reality it’s a bunch of people collaborating and making something and cutting things out because it doesn’t work, or because there’s no more room on the disc or whatever.

Lewin: These aren’t all weighty decisions that changed the narrative or changed the ideas in the game. Sometimes a cut room is just a cut room.

Cifaldi: Right. If we start acknowledging that decisions are made for reasons, that things get cut for usually the right reasons, that we can sort of move focus away from these tiny details in the game’s development and start talking more about the process and what made the game unique and how the systems talk to each other and how decisions were made based on our knowledge now of how the game actually works. When I play a SCUMM game now, I feel like I’m in the Matrix. I understand everything that’s going on under the hood now. It helps me understand why decisions were made. Why this flame over here isn’t animated, why the screen scrolls in this particular way. It gives me this intimate relationship with the game that I could have any other way. I’m pretty thankful for that, and I’m excited for other people to experience that too.

GamesBeat: Where was this cut room?

Cifaldi: It’s on Monkey Island. It’s basically a connection to the cannibal village. From the overhead map, you would click on the cannibal village, but before it took you to the village, it took you through this path that upped the tension a bit. At this point in the game, all you know is they’re cannibals. You don’t know that they’re goofy cannibals that aren’t going to harm you. You’re walking through a path and seeing gore and horror and getting scared because you’re about to go to a cannibal village and they might kill you. It’s just a screen that’s there with no purpose other than adding tension, I think. You can’t do anything except walk through it. The only interaction is that when you get near the village, you can look at it, and he says something like, I can’t see anything from way back here. That’s it. It’s just a room you walk through to get to the village.

I mean, this is a world that’s existed in our heads for decades. It’s cool to flesh it out a little more. There are parts in the code that suggest to us that the developers thought fondly of it. It’s not something where they’re like, ah, kill it. I think it’s in the script for the overhead map. The part of the code where you click on the hotspot to go into the cut room, it’s still there, but it’s commented out, so it’s not compiled into the game. Then there’s a comment next to it that says, in memory of the unforgettable dripping leg. Something like that. They thought fondly of this room. It could be cut for various reasons. It doesn’t really do anything for the game. It just slows it down. That’s one reason. But the other reason is that the biggest use of disc space was art, and this was a ton of art. It was not just one giant room. It was also eight frames of leg dripping animation and four frames of smoke animation. It was a lot of art. It might have been something that was cut for that reason. Incidentally, I have no reason to believe, having investigated this code, that a closeup of the dog was ever a thing in the game. It’s on the back of the box. I think it’s just a piece of art. I don’t think you ever talk to the dog and get a closeup in the game. There’s no evidence to support that. I think they restored something that was never there, is my take on that.

The RetroBeat is a weekly column that looks at gaming’s past, diving into classics, new retro titles, or looking at how old favorites — and their design techniques — inspire today’s market and experiences. If you have any retro-themed projects or scoops you’d like to send my way, please contact me.

By VentureBeat Source Link