Spike debug console

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
Following my ramblings in the speaker lights thread, some people have expressed interest in getting info out of the game. I've found a way that doesn't involve fiddly code injection, and am sharing it in the hope that it's useful to our community.

Legal disclaimer: not endorsed by Stern, you will void your warranty, proceed at your own risk, ymmv, etc.

Credit: I have to give credit to the guys at Mission Pinball Framework who have not only done a lot of hard work here, but also written excellent documentation, which definitely saved me some time along the way. This page is the interesting one for our purposes. Note the photo on there shows an older spike boardset -- on mine the CN2 connector is placed closer to the center, next to the main daughtercard connector (see attached photo).


Debug menu​

Spike game executables have a debug console menu built-in. (NB: I've only tested IMDN and BK3 but I'm pretty sure they all have it. BK3 had more options available, maybe because it was released a year later?)

I'll share examples of the menu in the next post in this thread. Please share examples from other titles if you can!

By default the debug menu isn't wired to any console (tty), so it is effectively disabled.

To access it, we need to do 2 things:
  1. Get a serial port connected to spike CPU.
  2. Configure the game executable to be wired to this serial port.

Can i haz cereal?​

The MPF documentation I linked above explains this in great detail, so I will summarise the 2 options here:
  1. Manly soldering skillz: solder a 3-pin 0.1" header to CN2. Note that the GND pin will sink a *lot* of heat (even my 80W iron needed a while to get things flowing) -- you could also skip that pin and grab GND from one of the unconnected connectors.
  2. USB-to-serial device (aka "FTDI adapter") plugged into one of the USB host ports.
Personally I went for option 1 because it's cheaper and neater, but option 2 has the advantage of being portable between machines.

Whichever way you choose to go for, you'll need a(nother) USB-to-serial device to interact with the console port. Unless you hook it up to something like a ESP32's UART of course.

Configure the game's console port​

For this you will need to edit a text file on the SD card. This isn't possible from a Windows PC. It might be possible from a Mac - I don't have one to try with. If you don't have a Linux PC with a memory card reader handy, create a bootable USB stick and boot from that to temporarily get a Linux computer without touching your existing hard drive. Some docs for that here or here.

Once you've got this, navigate to the 352MB partition (it's the rootfs) and edit this file:
Code:
spk/packages/spike-2_0_18/etc/init.d/game_console
changing both lines to point to /dev/ttymxc0 instead (this refers to CN2).

If you went with option 2 above (USB-to-serial device), then make it /dev/ttyUSB0 instead.

Note the version number (2_0_18) may be different, and there may be other spk package versions present. Most likely you want the highest number. If in doubt just edit all the files!

As mentioned I went with option 1, so my file contents are now:
Code:
CONSOLE_LOG=/dev/ttymxc0
CONSOLE_INPUT=/dev/ttymxc0

Usage​

If you're using CN2 you will get the output from the bootloader (U-boot) and some OS startup messages.

When the game boots, you will get a few warnings on the output (of course) and then nothing.

Press any letter then ENTER -- you should be greeted by the menu's help text. I'll put an example transcript in the next post.

What's it useful for?​

This depends who you are. I can see 3 uses but I'm sure people will come up with more :)
  1. You're a naff player like me. With this menu you can start any mode including the wizard modes which you'd otherwise never get to experience. They're usually the coolest ones too!!
  2. You're a mod maker. With this menu you can get some limited info out of the game (modes status), and also trigger sound effects or display animations (they're opaquely numbered though, so good luck finding the one you need).
  3. You're a technician. With this menu you can automate some of the diagnostics etc. Not sure it's really useful in this case tbh -- let me know if you do something cool here.
 

Attachments

  • IMG_20210609_145046.jpg
    IMG_20210609_145046.jpg
    203.6 KB · Views: 27
Last edited:
  • Like
Reactions: Wizcat and stumblor
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
Here is what you get on CN2, from my BK:SoR prem running the latest code (1.02 iirc). Separated in 3 sections.

Bootloader and OS init (you won't get this if using method #2):
Code:
U-Boot 2015.04+fslc+g5d9ffd2 (Jun 19 2020 - 16:33:21)

CPU:   Freescale i.MX6DL rev1.3 996 MHz (running at 792 MHz)
Reset cause: POR
Board: Wandboard
I2C:   ready
DRAM:  1 GiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
*** Warning - bad CRC, using default environment

INIT: version 2.88 booting
Starting udev
Enabling LVDS panel... done.
ALSA: Restoring mixer settings...
Found hardware: "sgtl5000-main" "" "" "" ""
Hardware is initialized using a generic method
Found hardware: "sgtl5000-center" "" "" "" ""
Hardware is initialized using a generic method
INIT: Entering runlevel: 5
Starting system message bus: dbus.
Starting syslogd/klogd: done
checking network bridge code version...
Bridge firmware revision: 0.5.0 up to date, no update required.
udhcpc (v1.23.1) started
Sending discover...
Sending discover...
Sending discover...
No lease, forking to background
/etc/rc5.d/S95game: line 340: dropbearkey: command not found
/etc/rc5.d/S95game: line 343: dropbear: command not found
starting application...
INIT: no more processes left in this runlevel

Game startup:
Code:
(gst-plugin-scanner:606): GLib-GObject-WARNING **: cannot register existing type 'GstVorbisDec'

(gst-plugin-scanner:606): GLib-CRITICAL **: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:606): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
MAX resolution 1360x768
UPDATE: no update found, exiting
pthread_mutex_unlock(sys_irq_hdw_enable): No such device or address

Playing around with the menu aka the good stuff ;)

To aid readability I've prefixed my inputs with '> ' -- this isn't actually present.
Code:
 > f
"f": command not found.
commands:
        adj: perform operations on adjustments.
        amp: amp system interface.
        aud: perform operations on audits.
        bdl: start background display layer effects.
        coil: fire a coil.
        credit: manipulate credits.
        debug: get debug information.
        display: display system interface.
        dsp: DSP interface.
        knight_spellout: knight spellout state.
        knocker: perform operations on the knocker.
        le: lamp effect system interface.
        mode: interact with modes.
        node: node diagnostics commands.
        nv: get non-volatile ram info.
        proc: get information about processes.
        red: redemption interface.
        replay: replay interface.
        scene: get information about layered display effect managers and scene cache.
        scene_cache: scene cache system interface.
        score: manipulate player scores.
        sd: shield_diverter.
        sound: sound system interface.
        uplock: interact with upper playfield lockup device.
        video_display: video display system interface.

> mode status
Mode   Lit   CanStart   Running   Spotted   Complete   Completions
------------------------------------------------------------------
   0 |      |    Y    |         |        |           |       0    | (Burning Sands)
   1 |      |    Y    |         |        |           |       0    | (Mud Bog)
   2 |      |    Y    |         |        |           |       0    | (Molten Fire)
   3 |      |    Y    |         |        |           |       0    | (Wicked Cavern)
   4 |      |    Y    |         |        |           |       0    | (Deep Freeze)
   5 |      |    Y    |         |        |           |       0    | (Black Castle)
   6 |      |    Y    |         |        |           |       0    | (Catapult Multiball)
   7 |      |    Y    |         |        |           |       0    | (Triple Knight's Challenge)
   8 |      |    Y    |         |        |           |       0    | (INVALID)
   9 |      |    Y    |         |        |           |       0    | (INVALID)
  10 |      |    Y    |         |        |           |       0    | (INVALID)
  11 |      |    Y    |         |        |           |       0    | (INVALID)
  12 |      |    Y    |         |        |           |       0    | (WAR Hurry Ups)
  21 |      |    Y    |         |        |           |       0    | (Black Knight Wizard Multiball)
  22 |      |    Y    |         |        |           |       0    | (THE KINGS RANSOM)
  23 |      |    Y    |         |        |           |       0    | (SUPER CATAPULT Multiball)
  24 |      |    Y    |         |        |           |       0    | (RAGE Multiball)
  25 |      |    Y    |         |        |           |       0    | (Last Chance)

> uplock
usage:
        uplock status
        uplock light
        uplock lock
> sd
usage:
        sd test start
        sd test stop

> uplock status
Upper Playfield Lockup Device Details
-----------------------------
Locks Lit     = 0
Balls Locked  = 0

Device Locks Virtual = no
Device Locks Enabled = 0
Device Locked Balls  = 0

> mode
usage:
        mode status
        mode light N
        mode start N
        mode complete N
        mode reset N
        mode complete_all
        mode reset_all
        mode time <N> - add N seconds to all running timed/hurryup modes
 
  • Like
Reactions: Wizcat and stumblor

FlipOutNH

Registered
Jun 15, 2021
2
2
3
Manchester, NH, USA
I am using the USB->Serial serial->USB type of connection. for some reason, it keeps auto-entering the output as input so I get an endless stream of the "command not found" response along with the list of acceptable commands.
Any idea why it would do something like this without me entering anything on the keyboard?
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
I am using the USB->Serial serial->USB type of connection. for some reason, it keeps auto-entering the output as input so I get an endless stream of the "command not found" response along with the list of acceptable commands.
Any idea why it would do something like this without me entering anything on the keyboard?
It could be an issue with your terminal emulator, you should be able to disable echo in the settings.
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
Might have to pm you about that. Does it involve i2c sniffing?
The node bus is RS-485 with a proprietary message format. I sniffed it in software by code injection, but it should be possible to wire to the spare RJ45 port as well.
 
  • Like
Reactions: stumblor

stumblor

Site Supporter
Sep 16, 2018
2,459
2,813
113
London · Davey Price
stumblor-pinball.square.site
Alias
Davey Price
(May as well keep this in the thread for future reference)

I take it this the code injection technique, the MPF one?

https://github.com/missionpinball/mpf-spike

From a mod development perspective, you wouldn't really want users to be making these SD card changes themselves in order to enable a mod. The best scenario would be to supply a (pre-hacked) binary of the SD card for the game in question. Potential drawbacks with that could be copyright issues, and game settings being reset to factory?
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
No, MPF do something different whereby they replace the game executable with a proxy which is driven remotely from their python framework. Their goal is to write a new game and not use any existing code/assets/etc.

The code injection I did was intercepting the socket under the game's feet and decoding the node traffic. I also wrote a standalone executable which can run in place of the game. I haven't shared that publicly and don't plan to do so (so long as Stern are in business anyway :)).
 
  • Like
Reactions: stumblor

FlipOutNH

Registered
Jun 15, 2021
2
2
3
Manchester, NH, USA
It could be an issue with your terminal emulator, you should be able to disable echo in the settings.
I figured it out. The TxD and RxD were in a different order on the USB cable (my fault for not checking first). Got it working on BK3. Tried it on my AvengerIQ but did not get response. Also going to try it on IMDN, Star Wars and EHOH if only to see any difference.
 
  • Like
Reactions: Fubar and Paul
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
I figured it out. The TxD and RxD were in a different order on the USB cable (my fault for not checking first). Got it working on BK3. Tried it on my AvengerIQ but did not get response. Also going to try it on IMDN, Star Wars and EHOH if only to see any difference.
Glad you got it working :) Please can you report which games work as I've only tried BK3 and IMDN.

The best way to debug/investigate is to setup the MPU for remote ssh access. I might do another write-up on that...
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
the rumour is that this "feature" is going away on new code soon...
I don't really see the point - this is a useful feature and it doesn't hurt them in any way. I'm pretty sure they must have users for it too (their devs and QA surely, but maybe it's also available for operators?).

Anyway, based on @FlipOutNH's comment, it may be gone in AIQ already.
 

Neil McRae

Site Supporter
Jun 5, 2016
12,502
8,971
113
Surrey
playpinball.uk
Alias
Neil McRae
I think they are concerned someone does something that cooks the game and potentially the room it’s in.


Sent from my iPhone using Tapatalk Pro
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
Eh? I don't see how that's possible from this console. It's pretty limited, and they control it fully anyway. The spike electronics are pretty well designed and - I assume - should be robust to misbehaving software.

What they should worry about (imho) is protecting the licensed assets, because companies like Disney will give them trouble if those are misused/ripped. However they don't seem to care about pinball browser much, so ¯\_(ツ)_/¯
 
Last edited:

BigIan

Site Supporter
Jul 21, 2011
4,034
2,115
113
Belfast, Northern Ireland
Alias
ZIG
i think its more the latter, stern have to be shown to be trying to protect the assets of the license they have built their game around..

same reason stern keep putting out updates when cleland or the like update their code for a machine
 

tktlwyr

Registered
Jun 9, 2021
5
2
3
Pembroke Pines, FL
the rumour is that this "feature" is going away on new code soon...
I think it may be nerfed already. I screwed up something while changing permissions in Linux so I was forced to do a re-flash of of the SD card. Now the file path is etc/init.d/game_console as opposed to spk/packages/spike-2_0_18/etc/init.d/game_console. I know longer have the spk, packages or spike-2_0_18 folders. Changing the code in game_console to CONSOLE_LOG=/dev/ttyUSB0 and CONSOLE_INPUT=/dev/ttyUSB0 results in the game stuck on the initialization screen. Changing it back /dev/null allows the game to boot. Maybe it's something that I'm doing incorrectly..

Anyone feel like ripping the SD card from their IMDNLE? :)
 
OP
Fubar

Fubar

Site Supporter
Feb 5, 2020
550
390
63
Cambridge, UK
Alias
Thomas
Now the file path is etc/init.d/game_console as opposed to spk/packages/spike-2_0_18/etc/init.d/game_console. I know longer have the spk, packages or spike-2_0_18 folders.
I suspect this is somehow related to the game's update process. If you have an old SD card and update you'll get symlinks all over the place, but if you create a card from scratch then not. 99% sure it doesn't matter.

Changing the code in game_console to CONSOLE_LOG=/dev/ttyUSB0 and CONSOLE_INPUT=/dev/ttyUSB0 results in the game stuck on the initialization screen. Changing it back /dev/null allows the game to boot.
This happened to me once when the ttyUSB was not plugged in, and I suspect it's the game executable failing to init due to that. It could be that they've removed the FTDI driver in that build, or you're using a device with another chip like a CP2104.

The easiest method to debug this really is to get ssh access into the board. You need to configure the eth0 interface, and start dropbear from the /etc/init.d/game script .... I will send you some notes.
 

tktlwyr

Registered
Jun 9, 2021
5
2
3
Pembroke Pines, FL
This happened to me once when the ttyUSB was not plugged in, and I suspect it's the game executable failing to init due to that. It could be that they've removed the FTDI driver in that build, or you're using a device with another chip like a CP2104.

That's exactly what it was. I didn't think that it was getting hung up looking for the device but it was. THANK YOU!
 
  • Like
Reactions: Fubar