7-10 split? Fore!

Share this post:

Share on Bluesky Share on Twitter Share on Facebook

The Pico-8 is called a "fantasy console," a virtual machine that is built to specifications which simulate limitations and restrictions of real, retro hardware. Many times these consoles take a bit of a "what if?" approach. What if the NES could show more colors? What if the Sega Genesis/MegaDrive had twice the VRAM? Or the developer of the system might want to develop in retro charm and without the retro headaches.

One thing to keep in mind is that everything in a virtual machine is a decision by the developer. Oftentimes, decisions are made which enforce the retro aesthetic/modality over other concerns. The Pico-8's 32k limit on cartridge size was a choice. Its 8k "token limit" was a choice. Its particular flavor of Lua was a choice. These choices are carefully considered to allow developer flexibility, but still maintain that feeling of working within retro limitations. That tension is a lot of the appeal in working in this medium.

Having said that, I was hit with one of the less-exciting choices made for the system and this week scrambled to find a solution.

Who needs more than 256k?
(Me. I do.)

There are two flavors of Pico-8

The general application you can purchase from lexaloffle.com, which allows you to create software for the platform.
The exported console, which embeds a developer's code into a standalone Pico-8 runtime package for distribution.

Status Line is distributed for both flavors. The .p8.png file is the "cartridge" which anyone who owns Pico-8 can run. As well, I have standalone exported apps for macOS, Windows, Linux, and Raspberry Pi (because I can). These are all functionally identical.

Or are they?! (suspenseful_musical_sting.wav)

Sometimes certain details are forgotten over time, and this week I was reminded of a critical one. There is a crucial difference between the two flavors of Pico-8 when it comes to importing files. This is the mechanism through which games and save game files are loaded into Status Line; i.e. the drag-and-drop feature which makes the whole thing possible.

Drag-and-drop uses the `serial()` function of Pico-8 to interface with your dropped file. It is quite limited, offering no metadata about the file, no access to the file after the initial drag-and-drop, no arbitrary seek, no ability to go back to the start of the file after reading through part of it. It is a dumb pipe of raw data.

In the documentation there is a footnote under the `serial()` section which reads

[1] Channels 0x800 and 0x802 are available from exported binaries, 
but with a maximum file size of 256k, or 128x128 for images.

"exported binaries" refers to the standalone builds which bundle the code into a nice executable package for other systems.

256k...

Well that's just fine for the .z5 games; z5 game files max out at 256k, by definition.

But wait, I've also been testing .z8 games! Those are bigger. Much bigger; as big as 512k! I've been testing them just fine and oh my god... as an owner of Pico-8, during development I am not subject to the 256k filesize limit. And so I've been happily testing games that cannot be loaded into the standalone machine. In fact, this limit pretty much rules out running any z8 games of note, even Lost Pig (at 279k)! And z8 has been a big focus of my testing and bug fixing. Did I waste all of this energy on a dead-end?

Make like a banana

I was determined not to let this break my spirit, though I did feel pretty defeated as I puzzled over the problem. As usual when confronted with these limitations, we must think like a developer in 1983. My general rule of thumb when working on Pico-8 is, "If an Apple 2 or C64 could do it, then the Pico-8 can do it, too." This was why I took on the challenge of PicoCalc, a clone of VisiCalc which debuted on the Apple 2. Why should the Pico-8 be denied such toys?

Back in the day it was quite common for games to ship on multiple disks. Perhaps we can use that same strategy on the Pico-8? If we can create "multi-disk" versions of large .z8 games, with each "disk" being 256k or less, we should be able to load in large games. Well, in essence, the 256k filesize import limit of the Pico-8 parallels "disk size" limits of yore, just reimagined for the fantasy console generation.

To make this work we now need

A mechanism for splitting .z8 games into two, 256k chunks
A mechanism for loading a multi-chunk z8 game into Pico-8

Status Line Split

A simple web utility has been created which does the file split. It is brain-dead simple in its implementation, doing literally the bare minimum necessary to make this process work. The utility works completely offline, in browser, with local files. Nothing is sent to any external server; it's just a tiny bit of javascript. Select a .z8 file and receive that file split into two chunks.

Each chunk holds 256k minus 8 bytes for a total of 512k minus 16bytes (just a hair under true 512k)
An 8-byte header is added to each chunk which holds
- A 3-byte identifer that "this is a Status Line Split file" (hex code 0xDECAFF)
- A 1-byte identifier for "chunk 1" or "chunk 2" (integer 1 or 2)
- A 4-byte identifer for "these chunks are related" (the gamefile's built-in checksum value)

Everything else is just the raw bytes of the z8 file.

A round of golf

There is a concept called “code golfing” wherein a programmer tries to solve a problem in as little code as possible. The low character count of the solution parallels the low score golfers try to achieve. The process of shortening code is called “code golfing.”

I discovered my oversight here as I was just about to go to beta for the project. I had worked on lots of weird visual edge cases and annoyances, growing the code base as a result, pushing ever closer to the hard-and-fast 8092 token limit of the Pico-8 (this limit does not change between dev and distribution builds).

I was down to my last 28 tokens when i discovered this issue.

So I knew what needed to be done. I had sketched out a plan of attack for working the multi-chunk load mechanism into the existing code, and I knew I was going to exceed token allotment. But as they say, first make it work then make it good.

I removed the save game load function temporarily so I could write and test the multi-file game loading. Luckily I had already written the load process to append to and grow the "memory" (just a Lua table of bytes) of the z-machine flexibly. It adds "memory banks" that hold 64k of data as needed. The game only requires 2 banks? No problem. Need 8 banks? No problem. We have up to 2MB of Pico-8 RAM (minus various overhead) to hold data and for this we only need up to 512k.

So the real trick of the matter is detecting and handling the multi-file loading part. Did the player bring in part 1 of the game first, or part 2 by accident? After loading part 1, was the file indicated as part 2 *actually* part 2 of the same game? How do we indicate this issues to the player? What was a pretty dead-simple load mechanism now needs extra UI layers and logic handling. This was not hard, but it definitely had a side-effect.

When all was said and done on this new loading mechanism, the token count for the project exceeded the Pico-8 limit by about 50 tokens. In Pico-8 development, that is A LOT. And the splashscreen hasn't even been fully reinstated.

There are various posts around the web about saving tokens in Pico-8; there are a lot of tricks.

a = 5
b = 10
c = 15

can be shortened to

a, b, c = 5, 10, 15

for a savings of 2 tokens. Do this multiple times through your code and it does reduce readability a bit, but 2-tokens or more per condensation adds up. Well, it added up for me, anyway.

For this project in particular, I had fallen into a pattern of using ( ) to clarify my intentions (to myself) a lot. From Pico-8's perspective I was wasting tokens.

if ((someValue == someOtherValue) or (anotherValue == 42)) then

can become

if someValue == someOtherValue or anotherValue == 42 then

Trust in the evaluation precedence order! (and save 3 tokens in this example)

For z3 games, I use a long, empty string as a spacer between room name and player score in the status bar. I only used the string one time, in one function, for one specific type of game. In-lining the string saved 3 tokens.

And sometimes you just have to rethink your approach to a problem. As I referenced earlier, first we make it work, then we make it good. Once I had a working load mechanism, I was able to evaluate what was *truly* going on and what was *truly* important and reuse a pre-existing function to assist in the process. This saved a dozen tokens or so.

Crisis averted!

Now I'm closer than ever to moving into the beta stage. I have the additional burden of testing/debugging/finishing Status Line Split now, making sure that the games are truly working properly (though I feel great confidence in the matter, having investigated the final memory state and compared it to the non-split loads).

This list should be the final "what I'm delivering" package for v3.0