Friday, August 10, 2018

4 Hour Projects: Adding Video to Interactive Fiction

Overview

After listening to the Eaten By A Grue podcast:19 about the interactive laserdisc-based game for the movie "Rollercoaster", and then following that up with the episode about Hitchhiker's Guide:16, a spark popped in my head. It should be possible to somehow "watch" the Hitchhiker's Guide (H2G2) Infocom game and play clips of the TV show, movie, etc, syncronized to the scenes and what you're looking at.
All of the distributable code for this is available at the github page for the project.
Just about all of this project is based on existing stuff, but I glued it all together.
  • Frotz - runs the Z-code, in a text-based terminal
  • VLC media Player - plays back the video
  • H2G2 TV on DVD - the DVDs
  • Netcat - so my client can talk to the server
  • Tee - to split the output from frotz
  • Perl script (in this repo) - "reads" the text, matches text, tells VLC what to play
The whole thing kinda works, is buggy, unoptimized, but that's the nature of a 4 hour hack.

Architecture

This is the basic system... This was a quick sketch I made to get the idea down. Essentially there are two halves. (I also called it "zvid" instead of "llifvid". pls ign. thx.)

Server/Video player

The server on the left is essentially a perl script that:
  • Matches text from the interactive fiction
  • has a simple interpreted language that can perform sequential functions
  • outputs "remote" commands for VLC
The output of that perl script is piped into VLC. If you're going to reproduce this, be aware that enabling the VLC shell it was a bit tricky, and I didn't document the process. I think I needed to enable extended preferences in VLC to see the bit to turn on that feature.
Also on this side are the video files for H2G2 episodes 1 and 2. I used Handbrake to decode them off of my DVDs. I do know that the episodes exist online, but those may be set up for web streaming, and will need to be transcoded via handbrake or some other tool. It basically just needs to add some information in that isn't there.
Netcat (nc) is setup as a listener for the text output from the game engine. That is piped into the perl script, whose output is piped into VLC.

Client/Game engine

The client on the right is some bash script piping and connection of Frotz with the game file. I used the ms-dos version of the data file, although I do not know if any other versions differ in any way. This was just easiest for me to grab without digging out and setting up my Amiga to pull the files off of my game's floppy.
The output of frotz is piped through tee. Tee takes the output and splits it to two different places. First, is the console so you can see what you're doing, but it also usually saves it to a file. I've changed that path to be piping the output to netcat, setup as a transmitter to the server.

Interpreter

In the perl script is a quick interpreter that I hacked together to run little micro scripts of code. I started doing this as an array of arrays, but if you've ever done that in perl, you'd know that such things are never good ideas in that language. I briefly considered hopping over to python to do it, but I already had stuff done, and was still toying with the idea of having the perl script itself listening to a socket, which i was unsure of how to do in python. So it's in perl.
Anyway, I switched it over to be a plain text blob in the source file. At startup, it cleans up the text, removing comments, and empty lines. It also breaks it up as two elements per line; the opcode and the parameter, and store that as the runtime program. This vastly simplified runtime routines.
The two main entry bits for the language are the label and the text match. It's sort of event driven, sequential language, with no nesting, no calls, no iterations, none of that... one operation per line. I did include comments though which are denoted by pound sign, # and continue to the end of the line. They can be put anywhere, as they are filtered out before runtime.
: label
Labels are used for 'goto' statements, or calling the goto function to set the current PC (program counter). If you call the doGoto() function, it will adjust the PC to the line after that label, or to -1, indicating that it was not found, and there's nothing to do.
? text to match
This denotes text that should be matched. As the program runs, it reads in byte by byte from the client and accumulates it. When it hits a newline 0x0d, or 0x0a, it sends the accumulated text to the "got a line" function. That one tries to match each "text to match" string to see if it matches any part of the current line. If it does, it sets the PC to the next line, and returns.
From there, there are a few opcodes that can be called:
seek 100
This will seek the current video file to 100 seconds in.
until 110
This will wait until the timer hits 110 seconds in in the video. Due to limitations of time, this is implemented as a hack. Instead of looking at the video file to get the time, it remembers the last 'seek' number called, does a difference, and sleeps for that many seconds, blocking. For the above two codes the "until 110" call will essentially sleep for 10 seconds
done
Indicates that a sequence of opcodes is done. "do until done" will stop here" this leaves the PC at -1, indicating it's done.
player play
This sends the "play" command to VLC. It can send anything. Useful things are:
play    # press play on the current videopause   # toggle pause!frame   # advance one frame. forces a paused statestop    # stops the playerfullscreen on  # makes the video full screen (on|off)seek 100 # you can manuall call this as wellrate 2   # twice speed... or "0.25" for quarter speed. etcadd FILENAME   # adds a file to the playlist, and switches to it
These essentially just print out the command, as the VLC shell is consuming the commands directly.

Determining match strings

For this I basically ran "frotz hhg.z3" and copy-pasted long lines of text from the game to match that seemed unique. It worked okay but was kinda tedious.
I was originally going to extract out the room names/descriptions from the z3 files using the tools, but gave up on that for this simpler, quicker approach.

Determining timing

So to get the second counts, I ran VLC, playing the video file, directly: "vlc episode1.m4v", and did a lot of typing in the shell of the above commands. I would type "play" to let it play, then "pause" or "frame" to get it to stop. You can get the current time with "get_time", manually seek to specific times like "seek 300" or differences from the current time, "seek -10" for ten seconds ago, etc.
I could use the GUI for this, but it shows time as mm:ss rathe than time as seconds only.
It was tedious, but it worked. An easier to use mechanism for this would be advantageous.

Conclusions...

So yeah... it worked.  With a lot more work it could be made to be reliable, have a nice editor, and be easily streamlined into more games.  I'd personally want to see an integrated executable as well, to get rid of all of this netcat and shell weirdness, and just have one exe that runs, reading in a language file.  Game packages could be created with the language file, audio, video, etc.  But I feel like I've accomplished what I wanted to for this. 

No comments:

Post a Comment