Engine Chronicles

  • StillRiver was originally built to help us understand what OutFox was doing while it ran.

    That meant looking at frame updates, rendering, audio, input and the systems that continued working even when nothing had changed. The recent loader work expanded that view.

    Startup had become one of the largest performance problems in the engine. On bigger libraries, players could wait several minutes before reaching the music wheel. Once there, uncached media and repeated file checks could create further stalls during the first pass through a pack.

    We needed to measure those paths in the same way we measured gameplay.

    StillRiver now records more of the work involved in starting the engine and loading content. This includes directory scanning, parser activity, note-data processing, cache handling, media checks, audio-length detection and the different stages of building the song library.

    That gave us a clearer picture of where the time was going. We also now have a preference based block in the preferences, which we will keep into the releases for those that want to see what StillRiver records, it is all uncompressed TSV (tab seperated values) a spreadsheet basically). We we also provide some tools to read the new data, but we left them unencrypted so you can see what the game can collect, with full transparancy.

    Some operations were necessary to preserve compatibility with older charts and unusual folder layouts. Others were asking the same question repeatedly, opening files earlier than required, or running parser routes when there was no suitable content for them to process.

    Once those differences were visible, we could make narrower changes.

    The loader became significantly faster. Cache handling became more dependable. Theme wheels stopped repeating some expensive work, and several tempting gameplay optimisations were left alone when the measurements showed they could change behaviour.

    That last part is important.

    StillRiver does not exist only to justify removing code. Sometimes it confirms that a small change is safe. Sometimes it shows that a proposed shortcut would skip required work. In other cases, it gives us enough information to plan a cleaner replacement later without changing behaviour immediately. The same approach now applies across startup, gameplay and presentation, measure the current behaviour, find the work that is actually causing the problem, then change only what the results support.

    For players, this should mean faster startup, smoother first-time scrolling, more reliable cached content and fewer faults caused by unrelated systems repeating the same work.

    StillRiver began by showing us what happened during a frame.

    It now helps us understand the work required before the first frame of gameplay appears.

  • Project OutFox supports rhythm-game families with very different ideas about input, judgement, holds, mines, keysounds and chart timing.

    That means we cannot clean up gameplay by treating Dance as the default answer and forcing every other mode through it.

    Some modes care about fresh presses. Others also use buttons that were already held when a row crossed. Hold heads, checkpoints, mines and synthetic inputs do not behave identically everywhere. BMS, PMS, DTX and other formats also carry rules that cannot be reduced to familiar four-panel behaviour.

    Those differences are not edge cases.

    They are part of the game contract for each mode.

    Before we replace the older row and input paths, we are documenting those contracts properly. We need to know which inputs count, which rows can be judged, how held state is used, which effects belong to scoring, and which unusual behaviours are intentional compatibility rather than accidental engine faults. This also helps us improve and make the familiar four panel mode more robust.

    The duplicate mine judgement was a clear bug because the same mine received two applied decisions.

    A pre-held Pump hold head is different. So is Para/ds3ddx/maniax behaviour that reacts to held sensor input. Those may look unusual when viewed through Dance rules, but that does not make them wrong.

    The engine has to know the difference.

    Bomb notes (non-dance mines) are another example, some game modes have a system where a bomb note dies at the receptor, so there is no late hit chance, some are only active for 15ms before the receptor then they are inactive, whereas in dance, it has a symmetrical early late long system. this makes modes with the bomb notes not have a representation in the engine as they use the old mine system, which is unfair to those players and communities.

    We are not preserving old code merely because it is old. We are preserving behaviour when charts, players and game modes genuinely depend on it. Where the existing implementation is tangled, we can replace it with something clearer while keeping the result the same.

    This is the direction of the wider gameplay rebuild.

    Each mode should define its own rules instead of inheriting assumptions from whichever system happened to be written first. Shared engine services can then handle timing, input and row processing without flattening every game into one model.

    Pump should behave like Pump. Be-mu/po-mu and DTX should retain their own keysound and chart rules. Guitar, drums, Para, Dance and the other supported families should each receive the behaviour they require.

    That is how OutFox moves beyond being a collection of compatibility paths.

    It becomes an engine that understands the games it supports.

  • Some of the hardest bug reports we receive are also the shortest:

    “That judgement was wrong.”

    The player may be completely right, but the final judgement alone does not tell us where the problem happened.

    An input passes through several parts of OutFox before it becomes a result on screen. The engine receives it from a device, maps it to the correct panel, relates its timestamp to the song, selects a chart object, applies the rules for the active game mode, and then updates scoring, life, effects and theme behaviour.

    A fault anywhere along that route can make the final result look wrong. We are therefore designing a gameplay flight recorder for the newer timing and input systems.

    This is not simply a replay file. Its purpose is to record enough information to explain how OutFox reached a particular gameplay decision.

    That can include the input the engine received, how it was mapped, the quality of its timestamp, the chart position involved, which note or other object was selected, and why the resulting judgement was accepted or rejected.

    It will also help with cases that are difficult to reproduce from a normal replay, such as held-state decisions, repeated input events, row crossings after a delayed frame, or timing changes around warps and seeks.

    The first use is straightforward: inspect what a particular build observed and decided during the original play.

    The longer-term use is comparison. The same recorded input evidence can be passed through a newer build to check whether it still produces the same result. If it does not, we have a much narrower place to begin looking for the regression.

    This does not mean every unusual judgement is an engine fault.

    The recording may show that a backend only provided estimated timing, that fallback behaviour was used, or that the input genuinely fell outside the active timing window. The important part is that we can distinguish those cases instead of trying to reconstruct them from a screenshot or a description after the event.

    The gameplay flight recorder is still planned work, but all the StillRiver hooks are in place as we use them for debugging and testing at the moment, so it is a work in progress, rather than a completed player feature.

    Its purpose is to make difficult timing and judgement faults easier to reproduce, explain and fix, without relying on either the player or developer to guess what happened inside the engine.

  • Recent work on hold notes has focused on cases where the judgement was correct, but the note did not finish cleanly on screen.

    Different game modes expect failed and completed holds to behave differently.

    In Dance-style gameplay, a failed hold should continue moving through the receptor. The body and tail remain visible so the player can see the hold finish, even though it has already received an NG result.

    Pump-style holds need different handling. Once a hold has completed successfully, it should not leave the head, tail or another small visual effect sitting at the receptor after the result has been decided.

    We found that some render-side state could remain active after the hold judgement was final. The gameplay system knew whether the hold was OK or NG, but the visual path could still behave as though the hold remained in progress.

    That responsibility has now been made clearer.

    While a hold is still active, the renderer can continue anchoring and drawing it normally. Once the result is final, the visual behaviour follows that result.

    A successful hold can finish and disappear correctly. A failed Dance-style hold can continue past the receptor so its failure remains visible.

    Pump checkpoint holds needed an additional fix, these holds use checkpoint behaviour along their length. The final checkpoint at the tail could still trigger a flash after the hold had completed, leaving what looked like a tap or end-tip sprite at the receptor.

    We now suppress that final tail flash without removing the valid checkpoint effects earlier in the hold. The same investigation also returned us to an older geometry issue between the hold cap and body.

    There was a one-pixel overlap that was usually hidden by opaque noteskins. It became much easier to see with transparent or semi-transparent graphics, particularly with Reverse, MineHolds and longer mine objects.

    The cap and body generation has been adjusted so the pieces meet cleanly without cropping the cap texture.

    None of these changes affect scoring or judgement.

    They make the visual result agree with the gameplay result that OutFox has already produced.

    Completed holds should finish cleanly. Failed holds should remain visible where the game mode expects them to, and Pump holds should no longer leave small visual ghosts behind at the receptor.

  • Recent work on Select Music has focused on Simply Love FunZone and soon to be Infinitesimal.

    Both themes make heavy use of Lua to control how their music wheels look and behave. That gives them much more freedom than the standard engine wheel, but it also means the theme and engine can end up repeating some of the same work.

    A single wheel movement may update the selected song, request grades, refresh chart information, load banners and CDTitles, update previews, rebuild visible rows and move several theme actors.

    None of those jobs is unusual on its own, however, the problem appears when both the theme and engine keep asking for the same information while the player is scrolling.

    FunZone gave us a useful place to improve that boundary.

    Its newer wheel work uses a hybrid approach. Lua continues to control the presentation, while the engine provides clearer information about the visible wheel items and their grades.

    This reduces the amount of song-wheel state that the theme has to rebuild for itself.

    It also exposed several problems that became more obvious during fast scrolling.

    Grade updates could trigger recursive command loops. Delayed banner and CDTitle loads could finish after the player had already moved to another song, causing media from the previous selection to appear in the wrong place.

    Those loads now check that the selected item is still the one that requested them before applying the result.

    The preview notefield also needed to retain its validated layout while the wheel changed selection. That path has been tightened so refreshing song information does not disturb the preview presentation.

    Cold image caches created another source of stalls.

    When a banner or movie thumbnail had not yet been cached, generating it during rapid scrolling could briefly hold up the wheel. We have adjusted those paths so cached media is preferred and expensive work is less likely to happen directly in the middle of movement.

    We added a new Lua song and group wheels, but parts of the standard engine wheel were still active underneath. That created a hybrid system without clearly defining which side owned each part of the behaviour.

    We have started making that separation deliberate.

    Missing theme objects are now guarded more carefully, unchanged song selections do not repeatedly call GAMESTATE:SetCurrentSong, and repeated file checks are being reduced while the wheel is moving.

    We have also added lightweight measurements around the wheel so we can see which operations are causing delays rather than trying to judge everything from scrolling feel alone.

    The aim is not to slow the wheel down to hide the work.

    FunZone and Infinitesimal should remain responsive and keep their own presentation. The engine should provide the song, item and grade information they need without forcing Lua to repeatedly reconstruct it.

    For players, this should make Select Music less prone to stalls, incorrect delayed media and sticky first-time scrolling through large libraries.

    For theme authors, it gives us a cleaner boundary between the information supplied by OutFox and the way a theme chooses to present it.

  • As part of the continuing crossed-row work, we looked at reducing how much of the chart the engine scans during gameplay.

    The idea seemed reasonable. If we know the latest row where something changed, the engine should be able to stop its update there instead of continuing farther through the chart.

    StillRiver showed that it was not that simple.

    During testing, meaningful gameplay work was still happening beyond the proposed limit. In some cases, the latest changed row was behind the current tap iterator, the mine iterator, or both.

    Capping the scan at that point would have saved work, but it would also have skipped behaviour the engine was still expected to process.

    Those row paths currently handle more than note positions. They can also advance iterators, apply misses, process mines and hold checkpoints, update combo state, send theme messages and run mode-specific behaviour.

    That does not mean all of that work belongs together forever. It does mean we cannot separate it safely by assuming that rows are always processed as one simple forward range.

    Warps make this especially awkward. A warp can move the chart position forward without normal song time passing through every row. Fake regions and other timing rules can also contain rows that should not be judged in the usual way. Some game modes and converted charts add further exceptions.

    A future row service therefore needs more information than a start row and an end row.

    It needs to know how the engine reached that range, whether the rows are judgeable, which gameplay rules are active, and which remaining effects still need to be processed.

    The first proposed optimisation did not have enough information to guarantee that, so we did not use it.

    That is one of the useful things StillRiver gives us. It does not only show where work can be removed. Sometimes it shows that an apparently obvious shortcut would change gameplay behaviour in ways that are easy to miss.

    The current row path still needs untangling, but we now have better evidence about what it is carrying and where the boundaries cannot yet be drawn safely.

    A faster scan is only an improvement if it produces the same gameplay result.

    This one did not prove that yet.

  • While reviewing the crossed-row gameplay path, we found another place where the engine could use the right information at the wrong moment.

    Some gameplay rules need to know whether a panel was being held when a chart row crossed the receptors. The older path could answer that by checking whether the panel was held when the engine processed the row.

    Most of the time, those moments are close enough to appear identical, however, they are not always identical.

    If a frame arrives late, the engine may need to process several rows together. By then, the player may have pressed or released the panel after one of those rows had already crossed.

    Using the current controller state could therefore apply a later input state to an earlier gameplay decision.

    We have started separating those two moments.

    For normal forward gameplay, OutFox now keeps enough input history to check whether the panel was held at the time the row actually crossed, rather than relying only on its state during the later frame that processed it.

    This is particularly important for held-over-mine behaviour and other crossed-row rules. Those decisions should be based on what the player was doing when the event occurred, not what the controller happens to report several milliseconds later.

    The older behaviour remains available where the newer evidence is not yet safe to use. Warps, seeks, reverse movement, autoplay and some legacy rules still need separate handling, so this is not being applied blindly across every path.

    The change is deliberately focused on ordinary human gameplay moving forwards through a chart.

    If a panel was held when the row crossed, the engine should recognise that, if it was pressed or released afterwards, that later state should not be carried backwards into the judgement.

    It is another small part of moving gameplay decisions away from frame timing and onto the actual song timeline.

  • While reviewing the old crossed-row gameplay logic, we found a mine being judged twice.

    The first hit came from a real input. The player pressed the panel while the mine was in range, and the engine handled it normally.

    Later, when the same row crossed the receptors, another part of the engine checked whether that panel was still being held. It found the same mine and generated a second hit for it.

    Same row, same column and same mine, judged through two different paths.

    Both paths exist for valid reasons. OutFox needs to detect a direct press on a mine, but some game modes also need to punish a player who is already standing on a panel when a mine reaches it.

    The problem was that neither path had final ownership of the decision.

    The direct input route could judge the mine, while the later crossed-row route could still find it and act as though nothing had happened. Each part of the engine was behaving reasonably on its own, but together they allowed the same object to be serviced twice.

    Our gameplay logs showed the sequence clearly. A real mine hit occurred first, followed by a synthetic held-panel hit for the same row and column.

    That gave us a narrow place to fix it.

    The crossed-row mine check now ignores mines that have already been judged or removed from play. Held-over-mine behaviour is unchanged, and the normal direct input path continues to work as before.

    The fix does not alter mine scoring or remove any existing mine rules. It only prevents two parts of the engine from making separate decisions about the same mine.

    This investigation also showed why the older gameplay path needs clearer boundaries.

    Input handling, row crossings, object selection and the resulting effects currently happen across several connected systems. When more than one of those systems believes it owns the same decision, faults like this become possible.

    For now, one mine receives one judgement, which feels like a fairly reasonable rule for a rhythm game.

  • The recent loader work has now reached a point where the difference is difficult to miss.

    One of our older benchmarks recorded 3,902 songs taking 791.658415 seconds to load. That is just over thirteen minutes. When we began the current round of testing, our active 3,722-song library was still taking 677.246704 seconds.

    After working through the loader piece by piece, that same library reached repeated runs of around forty-six seconds. A later validation run completed in 44.012848 seconds. There was no single switch responsible for that change. This was on a USB external 2.5" 5400RPM 2009 laptop drive we had lying around. On newer NVME/SSD architecture, the gains will be much higher.

    A large amount of the loading time came from work being repeated, or from expensive checks being used before the engine knew whether they were necessary.

    The first major improvement came from format detection. Ordinary SSC files were sometimes being checked as possible Pump content based on signals that were too broad. That preserved compatibility, but it also sent thousands of files through parser work they did not need.

    We tightened those checks so Pump content is still recognised where the files provide real evidence, while ordinary SSC charts can continue through the normal path.

    The loader was also asking the filesystem many of the same questions more than once. Once a folder had already been listed, that information could often answer basic media checks without probing the same files again.

    Chart data had a similar problem. Some load-time tasks were repeatedly compressing and rebuilding note data that was already available in memory. Those tasks can now use the existing chart data instead of recreating it during the same load.

    Audio length checks were another large cost. The older tidy-up path could fully open audio files while loading, even when the engine only needed enough information to complete cache data. On a cold startup across several thousand songs, that became extremely expensive.

    The newer loader path uses quicker audio length information where that is sufficient, while leaving full audio opening to the parts of the engine that actually require it.

    Several smaller changes contributed as well. Empty parser routes are skipped, PIU work is avoided when there are no candidates, cache hashing spends less time inside the write lock, and loader state that should belong to one worker is no longer shared across the whole process.

    None of these changes would matter if the faster loader produced a different library.

    Throughout the main validation runs, the song, step, Pump and autogen counts remained stable. The improvement came from removing unnecessary work, not from silently dropping content.

    There is still more loader work to do, but startup for this test library has fallen from more than eleven minutes to under one minute.

    That is a substantial change players with large collections should notice immediately.

  • As part of the loader work, we have also been reviewing how OutFox decides whether cached song data can still be used. Caching is essential for large libraries. Once a song has been parsed, the engine can save the result and avoid repeating the same work every time it starts.

    That only helps when the saved data still matches the files on disk.

    A chart may have been edited, a banner replaced, metadata corrected, or a parser changed since the cache entry was created. If the engine does not recognise that change, it can load stale information even though the source files are correct.

    We have been tightening those checks across song data, edits and media so that OutFox can reuse cached results when they remain valid and rebuild them when they do not.

    One fault uncovered during this work appeared as a difficulty meter of -2147483648.

    That value came from an invalid conversion rather than anything written by the chart author. The obvious fix would have been to clamp unusual meter values globally, but that would also damage legitimate content with intentionally high ratings.

    Instead, we corrected the specific conversion that produced the invalid integer while preserving the original floating-point meter data and authored values.

    Threaded loading exposed another problem.

    Some information that appeared local to one song load was actually being shared between several loads. Image fallback state and parts of the dynamic parser path could therefore produce different results depending on which thread reached them first.

    We have kept the parts of threaded loading that are safe, while restoring ordered processing around the older code that still depends on shared state.

    This is slower than declaring the whole loader parallel and hoping for the best, but it gives us consistent results between runs.

    For players, the cache should simply do its job.

    Unchanged songs should load more quickly. Edited charts and replaced media should be detected. Banners and previews should remain stable, and chart information should not change because the library happened to load in a different order.

    The loader work is not only about making startup faster. It also has to make sure the faster result is the same result we would get from reading the source files again.