In addition to the following report, further details of Nekotekina and kd-11’s work during September and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-09-14)
Status update from Nekotekina (2019-09-25)
Status update from kd-11 (2019-09-30)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

After the large improvements in August, September featured a lull in testing games as our regular testers were focused on testing various pull requests opened during the month. Due to this, most categories only showed modest changes. However, the Loadable category did decrease from 16 games down to just 14 games now.

On Git statistics, there have been 17640 lines of code added and 7957 removed through 89 pull requests by 14 authors.

Major RPCS3 Improvements

RSX improvements and fixes (#6483, #6488, #6572, #6667)

This month, kd-11 made a multitude of RSX improvements and addressed a number of bugs present in the graphics output. Let’s take a quick look at the major improvements from the list. First up, kd-11 restructured the command buffer to to allow faults in vertex upload and defers the renderpass open to allow recovery after a fault in the middle of vertex upload. This fixed scenarios where flushing during draw calls would lead to inconsistent object state and “recovery” draw calls would be guaranteed to fail or crash the driver. This reduced the flickering when Write Color Buffers was enabled in games such as NieR and also improved the rendering of few post-processing effects.

Next up, kd-11 looked into optimising RPCS3’s handling of textures. Firstly, he implemented faster texture upload by looking for opportunities to upload the full data block in one go as opposed to parts. Secondly, he implemented auto-matching of pitch dimensions. Analysing statistics from multiple games revealed that most most use pitch alignment of 1, indicating that there is no reason to force a 256-byte alignment when uploading. This allows fast-copy to work which should reduce the load on the GPU and CPU while also drastically lowering the memory footprint of smaller textures. Finally, compute and transfer operations are now batched when decoding texture data which greatly lowers the overall number of compute invocations.

Post-processing effects such as blur are now correctly displayed!

During this time, some users reported graphical issues in Metal Gear Solid 2 and 3 where Snake would be semi-transparent. This in itself was nothing unexpected, but with further user interaction, it was discovered that issue was present only with AMD graphics cards. This piqued kd-11’s interests and he began investigating into the issue. Initial testing revealed that this issue did not occur when MSAA was disabled and helped narrow the investigation to the implementation of the rendering pipeline search when using MSAA.

Before we proceed, here’s a quick look at RPCS3’s rendering technique. RPCS3 determines all the factors considered for a draw call and hashes them into a key (known as a renderpass key). These structures are then cached and fetched dynamically as and when the game requires it. However, the samples used in anti-aliasing were were previously not part of this key as they were assumed to remain static. After running hardware tests, it was discovered that it was indeed possible (albeit rarely) for a game to change the sample mask when rendering and such a change went unnoticed by the backend. This meant that not only the rendertarget configuration is important (with its renderpass key), but also the active sample mask. And as a complete surprise to nobody, Metal Gear Solid 2 was found to do this. Once the bug was out in the open, kd-11 promptly fixed it and Snake was once again visible in all his glory!

Thanks to the improvements made to the MSAA implementation, multiple transparency issues were fixed in Metal Gear Solid 2 & 3

Finally, kd-11 finished the month off with a couple stability fixes. Asynchronous flips are allowed to safely occur in the handle descriptor reset with thread::end during a begin-end cycle without crashing the drivers. Also, the width up from surface cache was aligned before bpp normalization to avoid a result of 0 width and fence/event timeout was increased to multiple seconds. This was necessary as it was possible for the driver thread to “starve” if the emulator is compiling LLVM modules with all threads, causing a small timeout to fail. This fixed a spurious crash in Gran Turismo 6 after the attract video.

sys_usbd Implementation (#5688)

Like any console, the PlayStation 3 had a host of accessories such as the Guitar Hero guitar controllers, Rock Band drum kits, SingStar microphones, Logitech Driving Force GT racing wheel and even toys such as the Skylanders Portal that could be connected for additional functionality. The PlayStation 3 had dedicated software libraries to handle communication with these accessories. The cellPad library was used to communicate with controllers such as the DualShock 3, Guitar Hero guitar, drum kit, dance mat and many more. Microphones such as the SingStar microphone utilised the cellMic library whereas devices such as the PSEye camera used the cellCamera library. The goal of these libraries was to streamline and standardise the interface that games used to communicate with various peripherals. RPCS3 has a HLE implementation of all these libraries with varying degrees of completeness that allow games to correctly recognise the peripherals connected to the PC.

However, this was not the only way games could communicate with accessories on the PlayStation 3. There existed an additional library known as cellUsbd which allowed games to directly access and communicate with USB devices without any management. This library was particularly beneficial for games that required functionality not available through the dedicated libraries. However, RPCS3 did not support this cellUsbd as it required an LLE implementation of sys_usbd (the lv2 syscalls used by cellUsbd) and was only used by a handful of titles. That is until GalCiv decided to take a look into it. If the name rings a bell, it’s the same developer who expanded cellPad to support various PlayStation 3 accessories in February and implemented cellMic in July. In his mission to expand RPCS3’s support for peripherals, it was only a matter of time till he set his eyes on sys_usbd.

With cellUsbd implemented, DJ Hero 2 now progresses ingame and correctly recognises the turntable!

Development had it’s bumps but after 6 months of tackling issues, the work on sys_usbd was finally completed and merged into the master build of RPCS3. The new USB system was designed to handle abstraction from the grounds up which means implementing USB hardware emulation is relatively simple. As a proof of concept, GalCiv also implemented an emulated version of the Skylanders Portal along with its figures and added a GUI Manager to easily select the figures during gameplay. Here’s a video showcasing Skylanders Giants that now progresses ingame thanks to this:

There is still work left to be done with sys_usbd such as adding support for streaming USB devices (used in few SingStar titles) which will be addressed at a later point. In the meantime, users who wish to play games that utilise cellUsbd can refer to the guide on our Wiki. Use of this library requires an additional setup process as both Windows and Linux do not allow applications raw access to USB devices for security reasons. Another important point to note is that the decision on which library to use is decided by the game. So a particular peripheral, say a GH guitar, may be accessed via cellPad by DJ Hero and via cellUsbd by DJ Hero 2.

Improvements by Nekotekina (#6471, #6602, #6631, #6637, #6645)

Continuing the trend of previous months, Nekotekina undertook a broad task of finding low-level bugs and errors in the codebase and replacing them with the correct implementation. As was the case with other developers, we don’t have time to cover all the changes made so here’s a summary of the major improvements made. Firstly, he completed the implementation of g_fxo as explained in the July progress report.

Next, Nekotekina focused on how RPCS3 handles unimplemented HLE functions. The previous designed required the emulator to throw an exception every time an unimplemented HLE function was encountered. Due to this, RPCS3’s codebase was plagued by exceptions. Work was previously done in the past by Nekotekina to reduce their negative influence of using exceptions, but proved to only be a stop-gap measure. Exceptions were originally used in RPCS3 as developers were unable to find a better alternative at the time, but it is now apparent that emulation has very little place for such heavyweight C++ features. It is especially detrimental with the JIT recompilers where exceptions can be thrown even when there was no good excuse to throw them. To minimise the use of exceptions, Nekotekina changed the behavior of unimplemented HLE functions to no longer throw exceptions but rather show up as a TODO in the log and return CELL_OK. Also, some native filesystem errors at RPCS3 startup are now tolerated, whereas previously they would throw and exception and crash the emulator. A trivial set of changes, but ones that will help push RPCS3 towards the goal of completely disabling the use of such features in the future. As a side effect, the TTY output can now be completely disabled by creating a directory called “TTY.log” in place of file!

Next up, Nekotekina tackled a few Windows specific issues. First up, the use of certain undocumented Windows syscalls for keyed events was dropped this month. Not much information is available for these syscalls, but analysis revealed that WinAPI begun abandoning the use of these syscalls around Windows 8. Nekotekina opted to replace these syscalls with native semaphores which have been available since Windows XP. After this, Nekotekina fixed an error in internal path creation function, where RPCS3 tried to create parent folders when they already existed. An attempt to do so would sometimes produce unexpected errors with the previous implementation.

Finally, Nekotekina finished his work of making the savedata saving function atomic. This greatly increases the reliability of creating user saves and provides safeguards against events such as emulator crashes, computer BSoDs or sudden power loss. With atomic savedata, the old savedata remains in place until the new data is successfully created. Once the new savedata has been created, a maintenance function will delete the old savedata. Within the same pull request, Nekotekina addressed one of the Windows-specific workarounds that was made to allow access to deleted files. On the original PlayStation 3’s filesystem, Linux and many other POSIX-compatible operating systems, deleting opened files is allowed and files will remain “invisible” on disk until the last user closes them. But on Windows, this feature is typically emulated by moving the file somewhere and marking it for deletion. Nekotekina changed the location of placing these files that are being deleted in order to not interfere with removing empty directories.

Games

Thor: God of Thunder

First up in this month’s roster is the console exclusive hack and slash game, Thor: God of Thunder. This title previously did not progress past Intros but was recently found to be fully playable!

Top Spin 3

The third installment in Sega’s Tennis series,Top Spin 3 is now playable on RPCS3. With this, both titles in the series released on the PlayStation 3 now run at full speed on the emulator.

NCAA Football 11 & 12

Finally, two titles from the NCAA Football series, NCAA Football 2011 and NCAA Football 2012, have been found to be Playable. Previously, these titles suffered from lackluster audio quality and were not considered to be playable. However, with the recent improvements made to the emulator’s performance, the audio no longer stutters and the games can now be comfortable played even on mid-range hardware.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

6471 – Don’t throw exceptions on unimplemented HLE functions. See coverage in major improvements here;

6484 – Fixed regressions caused by PR 6393;

6494 – Removed spin wait loop in cpu_thread::check_state and fixed an out-of-bounds error in the PPU Interpreter;

6503 – Fixed GCC compilation;

6504 – Improved ALSA backend;

6507 – Simplified more code using waitable atomics and implemented uniform wait abortion for named_thread;

6543 – Added few atomic fixes;

6560 – Updated Internal API;

6576 – Minor bug-fixes and maintenance;

6580 – Minor bug-fixes and maintenance;

6619 – Fixed a bug in cellSaveData;

6602 – Dropped using of undocumented Windows syscalls for keyed events. See coverage in major improvements here;

6631 – Fixed a bug in fs::create_path. See coverage in major improvements here;

6637 – Improved cellSaveData implementation to make savedata atomic. See coverage in major improvements here;

6645 – Complete the removal of fxm. See coverage in major improvements here;

6655 – Refactored endianness support;

6666 – Restored experimental optimized operators for se_t.

kd-11

6359 – Completed rewrite of DMA transfer handling;

6455 – Addressed static analysis warnings;

6467 – Changed implementation to not include read-only data when attempting to do section merge;

6483 – Restructured commandbuffer scoping to allow faults in vertex upload. See coverage in major improvements here;

6485 – Minor shader generation improvements;

6488 – RSX texture optimisations. See coverage in major improvements here;

6493 – Fixed OpenGL decode description for uint_24_8; Fixed broken source reference when doing Z scaling; Fixed missing target when using Vulkan surface transforms without scaling; Removed unused surface transform argb_to_bgra as Vulkan now has proper texture decoding;

6498 – Fixed implementation of fast texture copies when linear mipmapped images are uploaded;

6512 – Further fixed issues with the texture cache;

6534 – Addressed bugs in blit engine when moving inverted regions;

6535 – Improved use of CPU vector extensions;

6546 – Set SSE4.1 as target for accelerated index path;

6571 – Restructured flip code and frame scoping;

6572 – Fixed pipeline search when using MSAA. See coverage in major improvements here;

6594 – Avoided recursion in flip handler;

6644 – Fixed DEVICE_LOST error seen with Nvidia turing graphics cards when using the Vulkan renderer;

6654 – Added missing initialization;

6656 – Fixed scaling issues with the native overlay;

6663 – Restructured RSX deferred flip queue handling;

6667 – Allowed asynchronous flips to safely occur in the handle descriptor reset with thread::end during a begin-end cycle without crashing the drivers. See coverage in major improvements here;

6673 – Fixed missing point size export that addressed a graphical regression in Hyperdimension Neptunia;

6683 – Addressed few regressions with the OpenGL renderer.

Megamouse

6465 – Disabled a speculative check that broke controller vibration in several games;

6497 – Based on user feedback, increased the maximum setting for mouse acceleration to a factor of 30;

6416 – Added error_code and a lot of function parameters including proper log messages to most sceNp functions. This helps with debugging and further implementation. Also added some basic checks for these functions (some of which were wrong but got fixed already);

6545 – Fixed regressions caused by the above pull request;

6500 – Added error_code to cellGem, cellMouse and cellVdec. This improves log error messages;

6505 – Enabled the CLI arguments –stylesheet which loads RPCS3 with a custom stylesheet, –style which loads RPCS3 with the chosen system style and –styles which enumerates available styles. Also, added “None” to the stylesheet options in the settings dialog which disables any styles or stylesheets;

6591 – Fixed the enter button assignment in the native On-Screen Keyboard.
In RPCS3 users can select if they wish to use circle or cross as enter buttons in the native dialogs (the other one is used for cancel). This was broken for the On-Screen keyboard because it switched those buttons internally again after they were already switched by the overlay control utility itself, effectively rendering that option useless in that case;

6593 – Fixed a minor issue in Ninja Gaiden Sigma that caused the game to open the menu after saving a game if you held any button too long after dialogs. Hotline Miami had a different issue that was fixed by keeping the so called pad interception intact while there was still a button pressed after leaving dialogs. Since the pad interception flag is passed to the game we assumed that games just ignored input and did nothing else. But Ninja Gaiden Sigma will in fact open the menu whenever that flag is activated during regular gameplay, which is exactly what happened with the previous fix. As a solution, the emulator no longer keeps the pads intercepted after dialogs but simply doesn’t give the game any input while buttons are kept pressed when closing dialogs. This solution works correctly with both Hotline Miami as well as Ninja Gaiden Sigma. Also, removed some unused code in the pad utility;

6615 – Added the missing DualShock 3 button remapping and settings. Fixed an evdev regression caused by the above pull request that disabled a random button. Also, unified some of the pad handlers’ behaviors into their parent class and adds some virtual functions to simplify future implementations;

6638 – Addressed new compiler warnings caused by PR 6615;

6642 – Fixed the accidentally removed DualShock 3 pressure sensitive button presses in PR 6615;

6520 – Added some more error checks to sceNpTrophy. Ideally this should fix some random games, improve general accuracy and help with debugging by adding more accurate log errors;

6636 – Added a missing override in the keyboard pad handler that disabled the keyboard in the pad settings dialog;

6677 – Use the current config value for FPS in the game window title instead of initializing the game window with a fixed setting. This will be useful once the settings can be changed during gameplay in the future.

elad335

6482 – Fixed passive reservation locks checking at Emu.Stop() (non-TSX)
Fixed a race condition on the non-TSX path that could have resulted in SPU threads waiting for the PPU thread to unlock its memory lock when the PPU thread has already exited because the emulation has stopped. This was addressed by releasing the memory lock on emulation stop;

6583 – Allowed scNp2Init to initialize sceNp more than once internally. This fixed few regressions in sceNp2Init caused by PR 6545;

6604 – Don’t escape from check_state() before ensuring signal is resetted.
Fixed an old race condition which could have resulted in cpu_thread::check_state() to not reset all flags it should before it awakes. This problem was detected by regressions in games since recent sys_cond_wait improvements in because it was one of the places in code which relied on proper behaviour of cpu_thread::check_state().

6585 – Implement more mnemonics of PPU BC, BCCTR and SYNC instructions for the PPU disassembler. BC is the conditional branch instruction and has a lot of different variations based on encoding;

6587 – Since all cellSaveData functions operate through a single gigantic function called savedata_op() with multiple error checks, it is very hard to tell which error check failed based on parameters alone. This pull request added the official naming for failed conditions in error checks which provides more descriptive logs and eases debugging efforts;

6657 – Fixed error reporting in sys_rsx syscalls and disabled logging of “PSN offline” error codes in sceNp functions as it was impacting performance;

6595 – Optimizations of the two SPU instructions (CGX & BGX) for both SPU recompilers;

6661 – Fixed bugs in sceNpUtilCmpNpIdInOrder, sceNpUtilCmpNpId and sceNpUtilCmpOnlineId’s comparison operation and result;

6659 – Masked FIFO PUT on RSX execution;

6678 – Improved performance of FIFO PUT register masking which was implemented in the above pull request; Improved synchronization of rsx FIFO command semaphore_release.

RipleyTom

6476 – Reverted File.h encoding to circumvent an obscure GCC bug;

6393 – Simulate BSD File System for savedata file presentation order.
Megamouse discovered that Blazing Angels 2: Secret Missions of WWII gave a buffer for only one input file to the savedata function and thus made file presentation order critical. This makes file order the same as on the PlayStation 3 (first file written is presented first and deleted files leave an empty space for future files);

6557 – Updated minimum CMake version to ensure compatibility with Visual Studio 2019;

6567 – Changed implementation to report file size in bytes for better compatibility with the auto-updater;

5688 – Implemented sys_usbd (USB passthrough/emulation). See coverage in major improvements here;

6577 – Added back initialization checks back in sys_usbd which were incorrectly removed in the above pull request;

6592 – On Linux, Qt changes the conversion locale to that of the OS, which forces it to use OS locale default for numbers. Fixed games not showing that they had been updated in the UI (strtod() was failing because of different languages using different decimal markers ‘,’ or ‘.’);

6622 – Implement sys_usbd_event_port_send
Implemented a missing sys_usbd function (sys_usbd_event_port_send) which makes games calling cellUsbdEnd functional again;

6635 – Only free the list if libusb_get_device_list succeeded as gathering device list failed on some systems and crashed the emulator because of invalid ptr.

MSuih

6454 – Added disk cache removal to UI;

6511 – Added handler descriptions to gamepad settings;

6554 – Fixed Debug build compilation error;

6652 – Implemented game version logging.

CookiePLMonster

6538 – Make “Clear shader cache” clear all caches, not just the first one;

6524 – Added SCP extension support to the XInput pad handler.

jbeich

6522 – Dropped unused is_leaf_function;

6624 – Fixed compilation on BSD when using Clang 9.

JohnHolmesII

6440 – Added a message for the user when using a .rap file with its extension in uppercase (.RAP);

6519 – Fixed compilation warnings with clang relating to null pointer checks in sceNp module.

isJuhn

6438 – Added rounding_alignment argument to get_dir_size method to allow GetSizeKB functions to report size more accurately.

6480 – Removed SSE4.1 requirement.

windyboy1704

6559 – Corrected a typo in TSX instruction tooltip.

youwereeatenbyalid

6542 – Added a “stack mode” to the TTY output window.

arabek

6510 – Fixed compilation warnings found in sceNp module.

kevinxucs

6470 – Converted EROFS error to avoid crashes on a read-only filesystem.

Joaozin-tech

6437 – Updated the latest firmware version to 4.85.

Closing Words

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, MarioSonic2987, GalCiv, Megamouse, Digitaldude555 and elad335.

If you thought July was a big month for RPCS3, then prepare yourself for this progress report. This month saw 20 pull requests from eladash alone as he set a goal to have all his work merged in RPCS3 before his conscription service began. Not wanting to be outdone, Nekotekina and kd-11 also made a dozen pull requests each improving various parts of the emulator. We’ve got a lot to go over so let’s get straight into it!

In addition to the following report, further details of Nekotekina and kd-11’s work during August and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-08-16)
WIP Screenshots from kd-11 (2019-08-16)
Status update from kd-11 (2019-08-31)

Table of Contents

Important Announcements
Major Improvements
Games
Other Improvements
Conclusion

August saw a major change in the way games are counted in the compatibility statistics with all online multiplayer games in the Intro category and below being excluded. This was done to better represent the state of the emulator’s progress as these titles did not work as the servers that they wish to connect to no longer exist. Thus, even if RPCS3 becomes 100% complete, these games will not work unless somebody sets up a private server and emulates the original server accurately. These titles are still present in the compatibility list and are represented with a lightning bolt symbol next to their title. While they will still show up in all search queries, they will no longer be included in our statistics to showcase a more accurate picture of RPCS3’s development.

Moving on to actual improvements, our testers finally picked up the slack from the past few months and extensively tested the Ingame category which gave the Playable category its largest increase of the year. Every other category saw large decreases with Ingame shedding 88 games and Intro and Loadable recorded one of their largest drops ever percentage wise with drops of 17% and 20% respectively. An effort was also made to merge duplicated titles in the compatibility list which also aided the drop in these categories. As mentioned in the previous month’s progress report, the Nothing category doubled in size to 4 games due to erroneous tests, but this has since been fixed bringing the category back down to 2 games.

On Git statistics, there have been 8536 lines of code added and 5418 removed through 71 pull requests by 15 authors.

Important Announcements

Missing VCRUNTIME140_1.dll

For Windows users, if you encounter an error stating that VCRUNTIME140_1.dll is missing when trying to run RPCS3, please install Visual C++ Runtime 2019. With the move to using certain features available only with Build tools v142 in Visual Studio 2019, RPCS3 now requires the latest Visual C++ Runtime. Users can also find the link to the runtime on our Quickstart guide.

Major RPCS3 Improvements

Version increment to 0.0.7 (#6446)

In light of all the recent improvements made, RPCS3 has now bumped its version to 0.0.7! This has been long overdue considering the increase in compatibility and performance seen since January 2019. Apart from improvements to the core, we also saw the inclusion of support for PlayStation 3 accessories and the DualShock 3 controller, a native On-Screen Keyboard interface, various new themes for the emulator’s UI and the ability to increase frame caps using options such as Vblank frequency and Clocks scale.

Improvements made in compatibility since the previous milestone

Unfortunately, the full changelog is not yet available as it is a massive effort to document since it spans over 800 commits. However, users who wish to know all the improvements made to the emulator since version 0.0.6 can read our monthly progress reports starting from February 2019. A select few improvements were described in the Major Improvements section of every month while the full list of improvements available in the Other Improvements section.

Please note that the release marked on GitHub is only to mark the milestone. All users are recommended to download the latest master build from our website to get all the latest improvements and benefits.

Improvements made by eladash (#6286)

Anyone who follows RPCS3 knows eladash as a developer who has immensely improved almost every part of the emulator, especially the accuracy and performance of PPU and SPU emulation. Thanks to his work, a large number of games have become compatible with the PPU and SPU LLVM recompilers, performance on non-TSX CPUs has drastically improved and his lv2 & MFC contributions related contributions largely increased game stability. He is also known for analyzing issues plaguing games present in the Loadable and Intro categories, and helping them advance to the Ingame and Playable categories. In particular, some of his most notable cases is getting several Naughty Dog titles to progress ingame and figuring out the very last issue that kept Metal Gear Solid 4 from properly booting.

Unfortunately, eladash has to say farewell to the project as he has to fulfil his country’s military conscription requirements. This means that his contributions will be significantly curtailed from August till the end of his term. However, not wanting to leave things unfinished, elad made it a point to complete most of his contributions and have the same merged into RPCS3’s master repo. The end result of this was having 20 pull requests merged in this month alone, each one targeting a different part of the emulator. While we all love to see elad’s contributions to the emulator, it is simply not possible to cover all the contributions he made in the month. Instead, we’ve decided to focus on few important contributions that greatly benefitted us all. for the full list of contributions made by eladash, please see refer to the Other Improvements section.

First off, elad improved the implementation of various instructions relating to the PPU such as FCTIW, FCTIWZ, FCTID and FCTIDZ when the floating-point operand rounds to an integer bigger than INT32_MAX. Next, he fixed shift count in the PPU interpreter’s implementation of SRD and SLD instructions by masking the shift count with 127. While these improvements may seem minor, its impact was significant to say the least. Together with other improvements made in the pull request, Metal Gear Solid 4 went ingame for the first time on RPCS3! As a community, I don’t think we could’ve asked for a better parting gift than this.

In addition, elad also improved sys_get_current_time() to scale time relatively to emulator startup instead of Epoch to fix issues where the system clock would loop back after about 30 minutes to the beginning of timestamp when emulator was first launched.

MGS4 fixes and surface cache improvements (#6357, #6376, #6388, #6415, #6436, #6448)

Several weeks ago, while analyzing and fixing regressions, kd-11 encountered a particularly puzzling issue in newer Naughty Dog games, such as The Last of Us and Uncharted 3. Multiple user reports indicated that the graphics quality regressed and had a lot of ‘noise’ in their presentation. After spending days analyzing logs and traces, kd-11 realized that these were not actually regressions but rather issues that were always present in the games and hidden under other bugs. Now that they were out in the open, kd-11 decided to investigate these issues deeper and see if they could be fixed.

At first, kd-11 couldn’t find any possible solution, until he noticed that these games use the SPUs in addition to the RSX to render graphics which led him to theorize how a game would render with both Cell and RSX at the same time. He added a memory hash routine to scan large sections of memory for alterations and found a hit – the engine was rendering in parallel with both processors and re-injecting results from the previous frame into the last stage of the next frame. This meant that it was necessary to accurately initialize memory with Cell contents, a feature already planned, but largely unimplemented called “Read Color Buffers” and “Read Depth Buffers”. Several days later, this feature was implemented and the results were immediately promising. Users can find these options under the Advanced tab of the Settings menu.

In the meantime, kd-11 also received complaints of very poor performance in these games, and it was reported that using the “CPU blit” option seemed to help with this. For those unaware, CPU blit simulates the GPU copy engine on the CPU which is naturally inefficient but provides better accuracy. However, from the user reports it was clear that enabling this option yielded a performance benefit and was a clear indicator that something was broken. Investigating this further, kd-11 rewrote the blit engine behaviour to simulate a proper DMA controller and treating PlayStation 3 VRAM as completely separate memory from the main PlayStation 3 memory. He also added a DMA memory mirror pool with a sparse memory pool but this is still handled manually instead of relying on the GPU driver. This work drastically improved performance of the code and gave a few more frames in affected titles, including Naughty Dog games, God of War 3, etc. While the new blit engine DMA work is still not complete, its improvement to performance is already promising. Do note that in most affected titles there may not be much improvement in framerate if RSX is not the bottleneck.

Next, kd-11 created a PR dubbed “MGS4 fixes”, which implements 24-bit floats in depth texture decode step [D24S8 -> ARGB8], fixes some shader decompiler bugs and fixes behavior of varying registers. In the latter case, not all registers are created equally. Some registers have in-built saturate modifiers and texN registers can have their behavior altered through other registers. All tests that kd-11 wrote were confirmed on real hardware.

As the pull request title says, these changes fixes most graphical glitches found in Metal Gear Solid 4, as well as many other games. The depth issue which caused geometry to flicker in and out as well as being able to see through walls/characters has been fixed. Unfortunately, the issue causing shadows to appear completely black is still present as the fix for the issue caused too many regressions in other titles and had to be removed from August’s improvements.

Also, kd-11 made some improvements and changes, like implementing separable render target masking, fixing black textures in inFamous 2 and inFamous: Festival of Blood, some graphical issues in GTA V and a long-standing bug involving missing lifebars in JoJo’s Bizarre Adventure: All Star Battle.

Finally, kd-11 implemented GPU acceleration for texture decoding, which does benefit games that modify data a lot on the CPU. It also should help alleviate the performance impact of “Read Color Buffers” and “Read Depth Buffers” options.

g_fxo implementation (#6381, #6411)

With kd-11’s major contributions covered, let’s look at what Nekotekina has been up to. With a dozen pull requests merged this month, Nekotekina has been improving various core components and fixing few old regressions. We certainly can’t cover all of it here, so let’s focus on the removal of the fxm object manager in favour of g_fxo.

Before we understand what these object managers offer, let’s start at the top. A global variable is a variable with global scope, that is visible and accessible throughout the program. The set of all global variables is known as the global environment or global state. These variables greatly increase the ease of work, especially when many functions are involved. However, they are also considered dangerous as their values can be changed by any function that is called, increasing the chances of errors if misused by programmers. It also renders the overall code less modular and hence brought a need for an alternate solution. This is where objects managers come in as a replacement for global variables.

One such object manager, fxm was used to reset the global state automatically after emulation restart in a manner that prevented misuse by programmers. While it served the emulator’s functions well, it proved to be quite inefficient due to its use of reference counting pointers and the global mutex. In addition to this, it also had an overtly complex API that proved to be misleading. Nekotekina set out to find a more efficient solution to this function and designed g_fxo as a result. This new light-weight object manager is designed with a simpler API and provides an object that “just exists” without any synchronization overhead!

Once finalized, fxm was deprecated and replaced with g_fxo across all parts of the emulator that used it. While these improvements may not be readily visible in games today, it sets the stage for further improvements to the emulator’s stability. Users who wish to know more about Nekotekina’s contributions and their impact to the emulator, can read his status update on Patreon.

Screenshot function (#6334)

After adding support for the DualShock 3, microphones, guitars and various other peripherals in the past few months, GalCiv is back to add another heavily requested feature, native screenshot support. While we already have a plethora of methods to take screenshots of the games we play, they all are limited to taking screenshots at the screen resolution. With the introduction of resolution scaling in RPCS3, users have been able to increase the internal resolution up to insane resolutions such as 10240×5760 but the screenshots shared were sadly compressed down to the screen resolution which rarely exceeded 4K.

Thus there was a growing need to capture the screenshot prior to it being downscaled to the monitor’s resolution. This is where GalCiv jumped in and decided to capture the screenshot directly from the GPU’s framebuffer right before it is passed to the monitor. With a little guidance from kd-11, GalCiv was able to implement this feature in both OpenGL and Vulkan backend. Users can now press F12 to take a screenshot in-game (in the PNG format) which will be saved in the /screenshots/ directory.

Combining the new screenshot feature with the widescreen patch for Demon’s Souls makes for some amazing wallpapers!

More screenshots taken at native resolution thanks to this new feature.

UI Improvements (#6350, #6407, #6110)

RPCS3’s GUI has seen constant improvement throughout the year and this month is no exception. One major development is the new Advanced tab in the Settings menu to feature select settings that were either hidden within the Debug tab or not present in the GUI at all. This allows users to now easily modify settings such as “Vblank frequency” and “Clocks scale” from the GUI instead of manually editing game-specific custom config.yml files. With the new wave of 60FPS patches leveraging these setting options, the Advanced tab will certainly reduce efforts by the user to utilise such patches.

The next set of improvements are not aimed towards the graphical user interface (GUI), but in fact the exact opposite, the command-line interface (CLI). For a long time, users have expressed the need for a more robust CLI implementation in RPCS3. While RPCS3 did have a CLI, it was very limited in it’s capabilities and suffered from certain bugs preventing it from being truly useful. For a long time, this did not pose any issues as users were expected to use the emulator’s native GUI and only a niche group of users actually tried using the emulator through the CLI. However, as the emulator has matured over the years, there is a growing need to implement an automated suite that tests that would efficiently alert developers of any regressions caused by their changes.

But before we even think of attempting to create such a framework, the emulator should be capable of accepting commands through CLI or other non-GUI methods. To this end, Megamouse decided to improve the CLI and allow the emulator to work in a headless mode without any GUI or visuals whatsoever. However, since work on improving RPCS3’s CLI was being undertaken, the niche group of CLI users resurfaced and asked for a non-GUI mode to be added in additional to the headless mode. Both modes take the same approach but differ in their objective. The –no-gui parameter will not show RPCS3’s GUI but will allow games to launch with a game window whereas the –headless parameter will provide a mode completely devoid of visual output (no GUI or game window).

Along with this, the –hidpi parameter to launch RPCS3 with or without high DPI scaling (1 for enabled, 0 for disabled, default being 1). This does not override QT_AUTO_SCREEN_SCALE_FACTOR set in the environment, so users can now freely set their own scaling. The –version parameter was also improved to show the app version number. Click here for a full list of parameters accepted by emulator.

Games

Demon’s Souls

Thanks to Whatcookie creating a FPS patch for Demon’s Souls, it is now fully playable at 60 FPS! You can find the patch and instructions on our blog post here.

JoJo’s Bizarre Adventure: All-Star Battle

For the longest time, JoJo’s Bizarre Adventure: All-Star Battle ran with excellent performance but was not categorized as Playable due to a single graphical bug, missing lifebars during battle. After over a year, kd-11 finally managed to pin down the issue allowing this title to finally be considered Playable. All in all, It’s a good time for JoJo fans!

Crysis 3

But can it run Crysis? Well, kinda… In August, Crysis 3 progressed ingame for the first time and marks the milestone of all titles from the Crysis series being capable of progressing ingame on RPCS3. However, this is Crysis 3 we’re talking about, so the game is currently extremely unstable on RPCS3 and suffers from serious graphical issues as well. Soon

American Football and Rugby

This month saw 4 rugby and gridiron titles improve in compatibility. With most of the graphical issues resolved, NCAA Football 14 and Tecmo Bowl Throwback were found to finally be Playable! On the other hand, All-Pro Football 2K8 and Rugby League Live 2 went ingame for the first time. Sadly, All-Pro Football 2K8 suffering from low performance and a few graphical bugs while Rugby League Live 2 currently suffers from severe graphical issues. Nonetheless, it’s exciting to see how these exclusive titles will improve in the coming months!

Deception IV: Blood Ties & The Nightmare Princess

Another one for the weebs! The PlayStation exclusive title Deception IV: Blood Ties and it’s expansion pack Deception IV: The Nightmare Princess were tested and found to also be playable. These titles went ingame as early as 2017 but suffered from low performance preventing it from being playable. Thanks to the recent improvements to the emulator, both titles run at full speed on mid-range hardware.

Car Racing and Combat

Up next is a nice dose of car racing and combat titles. First up we have two racing titles, MotorStorm RC & Ben 10 Galactic Racing that are now fully playable on the emulator. But if weapons strapped to vehicles is more your style, Full Auto 2: Battlelines & Calling All Cars! were also found to be fully playable this month with minimal issues!

Mobile Suit Gundam: Target in Sight & Armored Core: For Answer

Looking at mecha titles, Mobile Suit Gundam: Target in Sight was found to be playable with good performance and graphics bumping the number of Playable titles in the Gundam series to 3.

Armored Core: For Answer

This title previously suffered from a physics bug where the character would endlessly fall through the ground upon progressing ingame, keeping this title in the Intro category. Thanks to the recent accuracy improvements made to the emulator, this bug has now been fixed and the game progresses ingame and has surprisingly strong performance. The game also seems to render graphics well, with the exception of the character model. A user over at our forums, makotech222, managed to finish the game on the emulator without a single crash. Further testing is necessary to determine if this title is indeed playable.

Journey: Collector’s Edition mini-games

Those who purchases the Journey: Collector’s Edition got thatgamecompany’s Flow and Flower in addition to Journey on the single Blu-ray disc. But that’s not all, the disc also included 3 never-before-seen mini-games from the studio’s 24hr Game Jams: Grave Diggers, Duke War!! and Nostril Shot. These 3 titles neither released on any other platform nor released as stand-alone games. Upon recent testing, all 3 titles were found to be fully Playable on RPCS3! While are small titles, their successful compatibility reiterates the founding goal of emulators to preserve games that would otherwise be simply lost in time.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

6293 – Added fallback implementation for waitable atomics that may improve performance with macOS and BSD;

6312 – Added new library loading modes: Load liblv2.sprx and manual selection and Load liblv2.sprx and strict selection;

6368 – Fixed regressions caused by PR 5883 and fixed a potential bug in typemap class;

6381 – Deprecate RPCS3’s object manager fxm in favour of a newly designed lightweight replacement known as g_fxo. See coverage in major improvements here;

c43a – Fixed a regression in VFS initialization;

6394 – Switched AppVeyor to use VS 2019 since VS 2017 advertised support for inline variables but had a bug preventing its use;

6411 – Further removal of fxm in favour of g_fxo and other optimisations. See coverage in major improvements here;

6421 – Fixed regressions caused by the above pull request;

6423 – Fixed regressions caused by the above pull request;

6425 – Fixed an emulator crash on Linux when the Create PPU cache option is selected in the Game list;

6444 – Fixed VFS filename escaping technique to allow control characters in filenames such as newlines. Windows does not allow the use character such as “:” while other operating systems typically do;

6450 – Fixed an issue with system time wrapping around about every 30 minutes on Windows caused by a 64-bit multiplication overflow. This was fixed by correctly using 128-bit intermediate multiplication results.

kd-11

6314 – Improved image_in implementation by correcting a typo (scale_x -> scale_y) and removing convoluted temporary buffer creation to just use vector allocator instead;

6325 – Fixed minor regression caused by the above pull request;

6338 – Fixed broken support for rgb pack16 formats; Added rgba444 transport values required by After Burner Climax;

6357 – Improved RSX surface cache. See coverage in major improvements here;

6376 – Fixed GL flushing of D24S8; Fixed FragDepth register initialization to match PlayStation 3; Improved memory usage by freeing ‘storage’ blocks if they get inherited by an actual framebuffer; Fixed OpenGL deadlock when using WCB and loading final framebuffer from CPU; Fixed stale references to invalidated surfaces in texture cache. See coverage in major improvements here;

6377 – Fixed offloader deadlock by requiring all spinlocks to invoke a wait callback in the backend;

6388 – Implemented 24-bit floats in depth texture decode step [D24S8->ARGB8]; Fixed shader decompiler bugs; Fixed behavior of varying registers. See coverage in major improvements here;

6414 – Fixed semaphore timeout bug by adding pending flip requests as a reason to invoke the RSX local task handler and release the vblank semaphore;

6415 – Implemented separable MRT masks, fixing graphical issues in inFamous 2 and GTA V. See coverage in major improvements here;

6426 – Fixed surface cache hit tests by avoiding broken tests due to queue_tag being called before pitch is initialized, returning the actual memory range covered while exclude trailing padding in surface -> get_memory_range() and calculating coordinates in src with src_pitch, not required_pitch;

6436 – Fixed bug affecting MRT color write lookup and surface_target_a. See coverage in major improvements here;

6451 – Fixed regressions from PR 6357;

6448 – Implemented GPU-accelerated texture decoding. See coverage in major improvements here;

elad335

72fb – Fixed a regression in sys_timer_usleep;

6281 – Improved accuracy of sys_vm_invalidate by not zeroing out contents of targeted memory;

6274 – Allowed loading liblv2.sprx with “Manually load selected libraries” and “Load automatic and manual selection” library loading options;

6319 – Fixed stack memory state checking in vm::check_addr where boundaries may be “allocated” but not mapped (guard pages);

6330 – Fixed potential segfaults in the debugger when attempting to set breakpoints on non-executable memory. Also, implemented vm::page_executable which may be useful in the future;

6269 – Improved sys_ppu_thread_create implementation by removing Remove 1MB limit for stack size, treating 0 stack size argument as 4KB instead of 16KB and returning CELL_ENOMEM for 0xffff’ffff >= stack_size > 0xffff’d000 which could have overflowed in VM due to the addition of 2 guard pages and the alignment up. Fixed crashing in Akiba’s Trip 2;

6329 – Improved fs::remove_all to properly reflect the correct error code during entries removal failure;

6286 – Fixed FCTIW, FCTIWZ, FCTID and FCTIDZ instructions handling when float rounds to an integer bigger than INT32_MAX, and fixed shift count in SRD and SLD instruction’s PPU interpreter implementation by masking the shift count with 127. Also, made sys_get_current_time() scale time relatively to emulator startup instead of Epoch. See coverage in major improvements here;

6336 – Fixed regressions caused by the above pull request;

6340 – Fixed regressions caused by the above pull request that caused seizure breakdancing in Ratchet & Clank Future: Tools of Destruction;

6321 – Fixed nv0039::buffer_notify when encountering overlapping memory at all cases; Fixed overlapping transfers of nv3089::image_in and nv0039::buffer_notify when out_pitch != in_pitch || out_pitch != out_bpp * out_w and memory destination and source is overlapping; Fixed undefined behaviour in image_in when using negative scaling;

6339 – Changed the default option for Sleep Timers Accuracy to Usleep Only;

6347 – Added additional pattern to PPU analyzer imports. This fixed issues in The Bourne Conspiracy and Ratchet & Clank Future: Quest for Booty where the emulator would hang for an extended period during the PPU initialisation phase;

6346 – Improved sys_process_get_number_of_object and implemented lv2_spu_image. This fixed a crash during the loading screen of ModNation Racers;

6076 – Added new accuracy control for PUTLLUC accuracy setting with the non-TSX path;

6364 – Fixed possible out of range methods execution with RSX emulation as command destination register progresses above UINT16_MAX limit;

5883 – Fixed temporary awakening of the wrong threads using awake threads-list with FIFO protocol (because it doesn’t match with the native priority protocol of the scheduler) by making thread-lists scheduling atomic; Fixed state fragmentation of running threads queue state at the time of beginning to end of thread list wakeup;

5863 – Implemented FPCC handling on floating point operations for PPU Interpreter (precise);

6385 – Fixed mapping on vm::null (0x0) address (used in recovering from SPU access violations after emulation stop by allocating the unmapped memory, now working on any address from 0x0 to 0xffff);

6405 – Improved sys_cond_wait implementation to wait for the thread to be scheduled before relocking the mutex which matches with the kernel behaviour on the PlayStation 3.

Megamouse

6292 – Changed behaviour of “Open Custom Config Folder” option to also select the proper file in the explorer on Windows and in the finder on macOS in addition to opening the directory;

6322 – Updated Appveyor CI to use Qt 5.13. This allows RPCS3 to benefit from relevant internal bug fixes and improvements to High DPI scaling;

6326 – Added two additional options to change the icon colors in the Trophy Manager and Save Manager. These options can be found in the GUI tab of the Settings menu. Also, fixed wrong icon height in the Save Manager when no icon was found and few possible crashes in the Save Manager and the Trophy Manager;

6110 – Added the command-line options “–hidpi” and “–no-gui”. See coverage in major improvements here;

6335 – Improved the number of supported keys for US and German keyboards and added an option to select the keyboard type in the System tab. Support for more layouts can be added quite easily in the future; Fixed multiple issues relating to the use of keyboard commands involving the Shift key when using the basic keyboard handler. For example, games want to be passed information as “Shift+5 was pressed” instead of “% was pressed”. Previously, there was no easy way of knowing this as Qt would skip the second key and convey only that “Shift and another key was pressed to get %“. The same logic was also added for the Alt key. This fix is currently present only on Windows;

6383 – Fixed an issue where older games that do not use the APP_VERSION parameter in their PARAM.SFO showed Unknown version in the game list. In such cases, the emulator now falls back to using the VERSION parameter;

6206 – Implemented both cellSaveDataDelete functions to allow users to delete save data from the save data list. Upon selecting an entry for deletion, a dialog will spawn asking for user confirmation. If the user accepts, the directory of that entry will be removed and another dialog will confirm the removal. If something went wrong a dialog will alert the user in the same manner. The list will be shown again in all cases. This process will repeat until the list dialog was cancelled, so the user can remove as many entries as needed; Added the initial focused entry feature to the native save dialog. The list will now keep its focus on the same index while performing actions in the list, instead of resetting the focus to the first element with every user interaction; Also, fixed an issue with save data autoloading;

6406 – Fixed a crash that occurred when the save data list overlay was loaded with focus on an element that didn’t exist;

6407 – Added –no-gui parameter in order to boot a game in its game window without any other components of RPCS3’s GUI. See coverage in major improvements here.

JohnHolmesII

6369 – Fixed mismatched enum string formats in sys_config;

6375 – Fixed compilation error on Gentoo by reapplying old template depth to compilers;

6432 – Fixed warnings relating to improper string concatenation in rsx_decode;

6458 – Updated building instructions after PR 6394.

AniLeo

6360 – Removed the fallback font from the YoRHa theme as a workaround to a bug in Qt 5.13 where having both fonts causes stylesheets to use the fallback font even if the primary font is present, but uses the first font when there’s no fallback font.

6412 – Updated OpenSSL DLLs to 1.1.1 in Appveyor builds;

6446 – Bumped RPCS3 version to 0.0.7. See coverage in major improvements here;

MSuih

6324 – Addressed a few Qt deprecation warnings;

6350 – Added Advanced tab to the Settings menu. See coverage in major improvements here.

GalCiv (RipleyTom)

6302 – Fixed a bug in suspend+stop threads where threads that were suspended couldn’t be stopped which broke the Sly Trilogy launcher. Now cpu::stop flag is tested prior to testing cpu::suspend;

6334 – Added screenshot functionality to the emulator. See coverage in major improvements here.

Plagman

6355 – Fixed a bug in Vulkan suboptimal swapchain affecting X11 Linux desktop with RADV and Freesync enabled (compositing disabled);

6354 – Fixed a crash on Linux when using newer versions of KDE.

CookiePLMonster

6371 – Improved vibration with the XInput pad handler through the use of std::chrono to correctly measure time between calls and preventing u8 values from overflowing and producing false positives on newVibrateData.

Yahfz

6366 – Changed severity of SYS_SYNC_PRIORITY_INHERIT protocol from Fatal to Warning as it was not actually implemented on the PlayStation 3.

MarioSonic2987

6344 – Updated tooltip for the native UI to mention that only Latin characters are currently supported (English, French, German, Italian and Spanish languages). For more details, please read our February 2019 progress report.

xddxd

6299 – Changed log level of shader cache removal as Success instead of Notice.

maximilian578

6256 – Expanded support to Visual Studio 2019 Build Tools v142 and improved NMake CL implementation.

al3xtjames

4850 – Implemented support for AES-NI hardware acceleration to improve decryption performance.

Closing Words

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, MarioSonic2987, Megamouse, GalCiv and Digitaldude555.

July was an absolute whirlwind of development that saw 60 pull requests merged from both our regular developers as well new contributors. That’s almost 2 pull requests merged everyday! This month, Nekotekina focused on improving TSX performance while kd-11 implemented a second round of bug-fixes that improved multiple AAA titles. On the other hand, eladash ironed out new features to help games go beyond their existing framerate caps and GalCiv implemented microphone support to finally allow RPCS3 to better emulate SingStar and other similar titles. Ohh and let’s not forget the surprise progress made with Metal Gear Solid 4 as well! There’s a lot to cover, so let’s jump straight into it.

In addition to the following report, further details of Nekotekina and kd-11’s work during July and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-07-10)
Status update from kd-11 (2019-07-23)
Status update from Nekotekina (2019-07-31)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

For the compatibility list, it was another quiet month as testers were predominantly occupied with testing pull requests before they were merged. The Playable category remained the largest, while the Ingame category showed a small increase. A similar decrease can be seen in the Intro and Loadable categories, dropping the Loadable category to an all-time low of just 20 titles. Finally, the Nothing category doubled in size to 4 games, although it should be noted this was possibly due to an inaccurate report from a tester, which is to be corrected soon.

On Git statistics, there have been 7396 lines of code added and 3334 removed through 60 pull requests by 15 authors.

Major RPCS3 Improvements

GT Fixes 2 and clipping fixes (#6179, #6185, #6224, #6228, #6236)

The Gran Turismo series has recently been identified as a competent litmus test for RSX accuracy. kd-11 has frequently had to overhaul sections of code to properly render graphics in these titles as they are tied very intimately to PlayStation 3 hardware, making even the smallest of inaccuracies apparent. This month saw several pull requests come in attempting to fix various graphical corruption bugs and other oddities found in these games.

The first improvement came from CookiePLMonster, a prominent member of the emudev community, who tracked down the cause of headlights and taillights being incorrectly rendered blue in Gran Turismo HD Concept, an issue that was leftover from kd-11’s last set of GT fixes. This game was using the less common SET_BLEND_COLOR series of commands to render the taillights. Very few games use these, so RPCS3 was using RGBA for color decoding without realizing that this was incorrect, and that the colors are supposed to be decoded as BGRA. This swapping of red and blue channels caused the error in rendering.

On the same day, kd-11 opened a pull request, appropriately titled “GT Fixes 2”, as a follow up to his previous set of changes to correct the RSX behaviour late last year. Gran Turismo 6 in particular had some bizarre rainbow texture corruption caused by poor handling of non-linear textures, since these textures have 4 border texels that need to be decoded properly. Another element of this pull request was adding proper clamping to clip dimensions for framebuffer mappings, which solved the issue of blit requests being denied excessively.

Later in the month, kd-11 further improved the MSAA implementation in RPCS3, fixing a number of transparency bugs in games that used AA to handle transparent texture effects. After this, he focused on implementing other relevant AA methods, which in this case was Coverage Sample Anti-aliasing (CSAA). Gran Turismo 6 and Gran Turismo Academy used CSAA on foliage and would render broken graphics without the implementation of the same in the emulator. With CSAA now implemented, foliage was fixed in Gran Turismo 6, Gran Turismo Academy and also GTA V!

Finally, kd-11 implemented a viewport raster clip by combining viewport clip region with the scissor clip region. However, this caused regressions in some games because buffer clears were clipped to viewport. With some debugging, it was identified that the viewport only clips during rendering but not during clear operations. This was promptly fixed by not clipping scissor to viewport when doing buffer clears. These changes fixed the broken split screen rendering and out of bounds elements in games like Sly 3, Hotline Miami, Dragon Ball: Raging Blast, Initial D Extreme Stage and many others!

Microphone support (#5446)

Anyone who has been following our progress reports would be familiar with GalCiv’s journey to adding support for various peripherals in RPCS3. In the past few months, he’s expanded the cellPad implementation to add support for emulation of guitars, drums, DJ pads and Dance mats, and also implemented proper support for the DualShock 3 allowing the use of pressure sensitive buttons and motion controls in RPCS3. Now, he’s set his sights on the cellMic library that was used by the PlayStation 3 to accept audio from microphones. With a bit of debugging on how cellMic works on the PlayStation 3, GalCiv was able to easily implement a basic passthrough for audio input from any microphone recognised by the Operating System to the game. This was a straightforward approach that worked surprisingly well, allowing Karaoke Revolution, Guitar Hero 5 and many other titles to properly recognise audio input. In addition, users can now play all instruments in games like Rock Band, Rock Band 2 and all Rock Band track packs perfectly allowing these titles to be fully playable!

The final piece to the puzzle, vocal levels in Rock Band titles are now playable thanks to microphone support!

However, nothing is ever as simple as it seems and the same was the case with cellMic. While Rock Band showed great results, SingStar did not work with this approach. With a little debugging, it was evident that SingStar titles are coded to recognise only the SingStar-branded microphones and ignore audio input from any other microphone. After identifying the device ID for the SingStar dongle, GalCiv implemented an option to emulate the regular microphones connected as a SingStar microphone. This promptly addressed the issue and allowed the original SingStar to progress ingame! After this, certain users expressed their wish to use an actual SingStar microphone with the emulator as well. GalCiv implemented this option allowing users to connect an original SingStar dongle and use the same in RPCS3. In addition, support for using the original Rocksmith dongle has also been implemented.

Users can now connect multiple microphones and select the relevant method of emulation as well

This allowed multiple titles to progress ingame and adds yet another set of peripherals to RPCS3’s list of supported devices. However, users should note that these improvements only benefit titles that detect audio input using the cellMic library. Many titles from the SingStar series use cellUsbd, instead of cellMic, to directly communicate with the microphones. However, not one to leave a job unfinished, GalCiv is hard at work on implementing support for sys_usbd (lv2 syscalls used by cellUsbd). Once finalised, users will hopefully be able to use peripherals such as the Skylander Portal and SingStar microphones in many more titles!

It’s all about time (#6189, #6207)

For people interested in playing PlayStation 3 games at frame rates beyond their original targets, it’s time to rejoice! Thanks to some new features by eladash, it’s now possible to play a significant portion of the PlayStation 3 library at high framerates without altering any of the original game code.

The first of elad’s pull requests added the ability to configure the vertical blanking (vblank for short) frequency. On old CRT televisions vblank was a period in which the television would not be drawing anything onto the screen. This was necessary for electron beam to return from the bottom of the screen back to the top of the screen so that it could get ready to draw the next image to the screen. With older game consoles such as the Atari 2600 or the NES the beginning of the vblank period would generate a vblank interrupt which would signal that the current frame has finished. On these old consoles, the vblank interrupt was often the only way to tell that time has passed. As such, game speed was closely tied to framerate for games targeting these older machines.

Fast forward to the PlayStation 3 era, and game consoles all include high resolution clocks that can be used to tell the time with great accuracy. Many game developers opted to tie game speed to the passage of real time, which allows their games to keep correct game speed even when the framerate is unstable. However, most developers stuck to what they knew best and opted to tie the max framerate to the vblank frequency. For these games, simply changing the vblank frequency will get you correct game speed at higher framerate with no extra effort required!

Sadly, not all games can have their framerate unlocked in this way while maintaining correct game speed. Some games may have their game speed tied to framerate, while others may use another method to cap their framerate. However, even in such cases, the new vblank option can make the process of achieving higher framerates at correct game speed easier since the people interested in making FPS patches will now only need to correct the game speed for higher framerates instead of first finding out how to unlock the framerate and then fixing the game speed after that. This hybrid method is exactly what was used to allow users to now play Demon’s Souls at 60FPS and beyond!

Following the inclusion of configurable vblank frequency, elad added two more options to the emulator. The “Clocks scale” option allows you to change the speed of various clocks and timers, making games think that time is passing slower or faster. Another option was added to allow you to configure the accuracy of sleep timers. Over a year ago it was discovered that a real PlayStation 3 has extreme accuracy with the sys_timer_usleep call, and the implementation of the same in RPCS3 was made more accurate to match the PlayStation 3. However, making it more accurate requires spinning to increase the accuracy, which means that your CPU spends a lot of time not doing useful work. Setting the accuracy to “as host” will get you the original accuracy that RPCS3 had, possibly reducing CPU usage in some games, but also dramatically reducing framerate in some games that rely on sys_timer_usleep to be hyper accurate such as NieR. Finally, another option was added that extends the accurate handling of timers beyond sys_timer_usleep to some other syscalls as well. This setting is known as “All timers”.

In the current master of RPCS3, these options are all available under the new “Advanced” tab of the Settings menu. If you wish to configure the vblank frequency or adjust clocks scale, then you must right-click on your game from the game list and select “Configure”, since these settings can only be set on a per-game basis.

UI Improvements (#6178, #6204, #6218, #6233, #6235, #6264, #6271)

Finally, let’s go over some of the improvements made to RPCS3’s GUI. Apart from our GUI whiz Megamouse, this month saw contributions to the GUI from CookiePLMonster and MSuih. The first set of changes aimed to fine tune how the UI reacts to controller input. CookiePLMonster observed that a few UI buttons did not follow the standard UI behaviour by executing their actions on button press instead of reacting to button click (press and release), This led to an inconsistent experience and in some cases difficulty navigating through the menus. Not many users would notice such a subtle difference but for those who did, this issue has now been addressed.

While the above change may seem trivial, the concept did have implications in relation to how input is handled by games while system dialogs are open. The PlayStation 3 hides button presses from the game when a system dialog is active to prevent the game from recognising unnecessary input. In order to achieve that a pad interception flag is activated and passed to the game whenever it requests for pad information and the input data passed to the game is basically empty as well. Megamouse had implemented this in RPCS3 a couple years ago and to put it simply, it just works. Except it didn’t. As we found recently, Hotline Miami 2 would immediately activate a menu option after a system dialog was closed without any additional input from users. This was obviously not the expected behaviour as the game recognised the last input made to the system dialog. Thanks to debugging from CookiePLMonster, it was identified that the game did not receive new input immediately after a dialog was closed until Cross, Square, Circle, Triangle, Start and Select were released (L1, L2, L3, R1, R2, R3, direction buttons and analog sticks did not matter). In the previous implementation, input was resumed on button press instead of button click (press and release). Megamouse promptly implemented this accurate behaviour in RPCS3 to resolve the issue.

Next, Megamouse fixed issues with background images shown during shader compilations screen and game titles. The background images are recognised from the directory of the PARAM.SFO that was used to boot the game. However, where a game has updates installed, the image are taken from the the update directory in dev_hdd0/game/ instead of the actual disc directory. This ensured that we picked up the latest image available. But in a situation where PIC1.PNG isn’t present in the game update directory, we end up loading the lower resolution ICON0.PNG as a fallback. Since loading a lower resolution image is not preferred when the high resolution image is available in the disc directory, the emulator will now look for PIC1.PNG in the disc directory before using ICON0.PNG as a fallback. In the same manner, the game title was previously taken from the game update directory (if present) and shown in the game window resulting in titles such as “Beyond: Two Souls Update Data (EU3)” instead of “Beyond: Two Souls”. Now if a game is detected as a disc game, then the emulator will read the original disc’s PARAM.SFO again and reset the title to match the actual disc.

Previously, the lower resolution ICON0.PNG was incorrectly used when the PIC1.PNG was available in the disc directory

This month also saw an improvements to the Virtual File System dialog and Save Manager dialog. First up, let’s look at the work CookiePLMonster did with the virtual file system dialog. To bring the dialog in line with the rest of RPCS3’s UI, the Add New Directory and Reset buttons were replaced with “+” and “-” buttons, the Reset All button was renamed to Reset Directories, the Okay button was renamed to Save and a Close button was added to allow users to quit the dialog without saving the changes made. CookiePLMonster also added the ability to delete a single item from the list which was previously not possible and a confirmation dialog was also added for the Reset Directories button to prevent accidental resets.

A comparison showcasing the changes made to the Virtual File System dialog

The Save Manager dialog also saw some notable improvements thanks to MSuih. When a save is highlighted, it now displays an icon along with the details to the right of the list instead of a separate popup dialog. This allows for much quicker navigation of the Save Manager and provides for a smoother experience.

A comparison showcasing the changes made to the Save Manager dialog

Finally, Megamouse wrapped the month off with improvements to icon rendering and resolution in the game list, Save manager and Trophy manager for high DPI monitors. In addition, the trophy icons now display the same background icon colour that is used for the game icons. The controller image size in the Pad settings dialog was also fixed for both normal and high DPI configurations.

Improvements made to to icon rendering for high DPI monitors

The controller image size in the Pad settings dialog was also fixed for both normal and high DPI configurations

Games

Metal Gear Solid 4: Guns of the Patriots

The promised day has finally arrived! Thanks to the sys_overlay improvement made by eladash, the coveted PlayStation exclusive title, Metal Gear Solid 4, reached intros towards the end of July. But it wasn’t soon after that eladash made further improvements to allow the title to progress ingame!

NieR Replicant and Gestalt

Thanks to Whatcookie’s recent patch for NieR Replicant and Gestalt, both games are able to break past their previous 30FPS framerate cap! And since NieR isn’t a very intensive game achieving a locked 60FPS with RPCS3 isn’t difficult at all. Check it out in the video below.

Megazone 23 – Aoi Garland

Now let’s kick off with some Playable titles! First off is the PlayStation 3 exclusive title Megazone 23 – Aoi Garland. When tested back in February 2018, this title was graphically accurate but suffered from low performance keeping it from being Playable. Thankfully, these performance issues have been resolved and the game fares well even on low-end systems.

One Piece: Pirate Warriors 2

Unlike its sequel, One Piece: Pirate Warriors 2 never reached PC. However, users needn’t fret as it was confirmed to be playable on RPCS3 from start to finish this month. Sadly, the original One Piece: Pirate Warriors, which also missed out on a PC release, still remains ingame for now.

Mist of Chaos

The last playable title from this showcase is Mist of Chaos. This title previously suffered from significant graphical issues keeping it from being playable. Thanks to the recent improvements by kd-11, these issues have been addressed and this game is now playable.

SSX

While SSX has been progressing ingame since mid-2018, the PSN versions of the game would consistently crash during the intros. Thanks to the various improvements made to the emulator, the PSN versions also progress ingame. Sadly low performance and graphical bugs prevent this title from being fully playable.

WWE SmackDown vs. Raw 2011

The last entry in the SmackDown vs. Raw series, WWE SmackDown vs. RAW 2011 finally progresses ingame. With this, all titles from WWE series that were available on the PlayStation 3 are now at least Ingame in RPCS3.

Miyazato Sankyoudai Naizou Sega Golf Club

While it may not seem that impressive, this PlayStation 3 exclusive title was one of the first titles released in 2006 as part of the console’s launch. It progressed ingame for the first time this month. Further testing is necessary to determine if this title is indeed Playable on the emulator.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

6172 – Build with no-strict-aliasing flag for GCC and Clang compiler;

6174 – Fixed regression from PR 6168;

6147 – Improved SPU implementation with the TSX path by fixing “Preferred SPU Threads” option adding vm::temporary_unlock to various syscalls;

6210 – Fixed SPU Interpreter regression after the above pull request; Added experimental TSC frequency detection; Added forward declarations of vm::_ref_base and removed default AT = u32 in _ptr_base and _ref_base; Also includes a commit from elad335 that fixed a typo in the kernel explorer;

6211 – Fixed a regression caused by the above pull request;

6225 – Optimised SPU Recompiler memory consumption by not rebuilding trampoline for every function at startup which should address the Out of Memory error in some cases;

6237 – Expanded thread affinity mask to u64 and also fixed and noted __APPLE__ path;

6249 – Improved TSX compatibility by Implementing low_lock and vip_lock (for shared_mutex) and simplifying suspend_all implementation with the updated shared_mutex;

6267 – Implemented waitable atomics, used atomic wait in shared_mutex and semaphore; Corrected get_int_t to get_uint_t;

6283 – Improved implementation of waitable atomics; Fixed sys_config_get_io_event syscall name; Also includes commits from elad335 that improved sys_lwcond implementation, made sys_fs_closedir atomic and added EAGAIN check for sys_memory_container_create.

kd-11

6155 – Improved object reuse and lifetime management. Also, fixed regressions caused by PR 6112;

6162 – Fixed native pitch computation in OpenGL;

6164 – Fixed frameskip in Vulkan; Fixed image_in functionality to avoid use-after-free. Marked empty inline draws as such to avoid a divide-by-zero later; Fixed unsupported GL format error in GLTexture.cpp; Fixed a regression noticed in Gran Turismo 6; Made AMD hardware tweaks for driver compatibility;

6179 – Dubbed “GT Fixes 2”. Implemented texture decoding with border texels present and fixed rainbow colors in Gran Turismo titles. See coverage in major improvements here;

6213 – Fixed regressions caused by PR 5937 such as broken depth in some games such as One Piece: Unlimited World Red and Valkyria Chronicles;

6224 – Implemented separate viewport raster clipping. Fixed broken split screen and/or elements out of bounds in Sly 3, Dragon Ball: Raging Blast, Initial D Extreme Stage and Beyond Two Souls. See coverage in major improvements here;

6228 – Added support for CSAA transparency without multiple rasterization samples which fixed broken foliage in Gran Turismo 6 and GTA V. See coverage in major improvements here;

6236 – Improved buffer clear implementation to no longer clip scissor to viewport. Fixed regressions caused by PR 6224. See coverage in major improvements here;

6261 – Fixed aliased memory surface loss when doing intersection tests. When doing memory inheritance across the DEPTH<->COLOR barrier, it was possible for data to be lost due to incomplete succession. Fixed a graphical regression in The Last of Us caused by PR 5937;

6285 – Optimised conditional render syncs to avoid unconditionally flushing the queue unless the upcoming reference is contained in the active command buffer. This avoids spamming queue flush, which frees up resources and improves performance by evading a GPU hard sync. Improves performance in Ratchet & Clank Future: A Crack in Time and Resistance 3.

elad335

6159 – Fixed possible inconsistencies for sys_memory mem stats report;

6167 – Clamped fragment shaders address to the relevant 31 bits;

6168 – Refactored rsx_decode.h to explicitly truncate bitfields to the destination type where the new bf_decoder resolves smallest type possible at compile time and truncates the result; Fixed a typo with NV4097_SET_BLEND_COLOR::green16/red16() that led to a loss of 8 upper bits (using u8 instead of u16);

6177 – Fixed a regression from the above pull request;

6173 – Added ability to log last PPU HLE function executed and full registers (SPU/PPU) state after access violation; Fixed MFC tag status report for explicit mfc barriers commands which affects only status report because it is already blocking all tags; Improved memory ordering of cpu_init() a little;

6190 – Fixed Emu.Stop() by not attempting to join the vulkan device enumeration thread;

6189 – Implemented the ability to configure vblank rate. See coverage in major improvements here;

6184 – Fixed access violation address reporting on PPU executable memory
and allows to detect access violations that were previously silent before;

6239 – Fixed sys_rsx_context_iounmap partial unmapping with entry indexing when part of the targeted unmap area has already been mapped or unmapped into non-continues EA addresses;

6207 – Implemented support to modify time scaling of all LV2, PPU, SPU and RSX timers including syscalls’ sleep time. See coverage in major improvements here;

5345 – Prefetch MFC list elements to protect them from destruction by the previous elements processing in the GET list. This occured when the destination of the command overlapped the list entries. Fixed Calling All Cars, making it go from Intro category directly to the Playable category!

6262 – Fixed unmap key value in cellGCM by correctly handling non-contiguous unmappings and other miscellaneous improvements;

6268 – Fixed a regression from the above pull request;

6102 – Improved sys_overlay implementation of sys_overlay_load_module_by_fd and implemented special segment allocation when ppc_seg flag is specified. With this improvement, Metal Gear Solid 4 progressed to Intros for the first time!

6280 – Improve ORI and ORIS disassembly for PPU modules.

Megamouse

6201 – Fixed the outdated app versions of games by properly refreshing the game list after a patch was removed;

6204 – Fixed controller icons size in the pad settings dialog for both normal and high DPI configurations. See coverage in major improvements here;

6218 – Improved button click recognition after overlay dialogs ingame. See coverage in major improvements here;

6233 – Fixed background icons in the shader loading screen and game titles in the game window for disc games with updates installed. See coverage in major improvements here;

6245 – Allows users to select all labels in the About RPCS3 window using a mouse and keyboard. Also, allows users to open links in that window using the keyboard.

6271 – Added improvements for high DPI monitors in the game list, save data manager and trophy manager. Also, added the background icon color that is already used for the game icons to the trophy icons. See coverage in major improvements here;

Whatcookie

6148 – Added a workaround for slow PBO usage with the OpenGL renderer using Mesa drivers for AMD GPUs on Linux;

6196 – Fixed a case where float16_t was wrongly disabled on non-Vega AMD GPUs on Linux with the Vulkan renderer;

6230 – Fixed .BIN decryption filtering for case sensitive platforms in the Qt file dialog;

6284 – Clarified warning regarding the blackscreen issue when using RADV drivers with LLVM 8.0.0.

MSuih

6142 – Cleaned up a lot of code to decrease the warning spam during compilation, particularly with CMake and MSVC;

6219 – Fixed an oversight that reverted the custom game name to the original game name when adding a custom config in grid mode or removing one in grid or list mode. Also, fixed a typo that made it impossible to remove custom pad configs through the game list;

6264 – Revamped the Save Manager GUI by adding resizable game icons (as seen in the Trophy Manager) and showing the save data details conveniently beside the list instead of opening the rudimentary dialog that was originally implemented. See coverage in major improvements here.

CookiePLMonster

6178 – Fixed behaviour of some UI buttons that used to activate on button press instead of the commonly used button click (press and release). See coverage in major improvements here;

6185 – Fixed decoding of SET_BLEND_COLOR RSX commands which corrected the blue tail lights in vehicles in Gran Turismo HD Concept. See coverage in major improvements here;

6235 – Improved the Virtual File System dialog by replacing the Add New Directory and Reset buttons with “+” and “-” buttons, allowed deletion of a single item which was previously not possible, renamed Reset All to Reset Directories and added a confirmation message, renamed the Okay button to Save to be in line with the rest of RPCS3’s UI and added a Close button was also added, allowing to quit without saving settings. See coverage in major improvements here.

GalCiv (RipleyTom)

6171 – Added an SPU performance test (spurs_test.self) to the elfs shipped with RPCS3. It benchmarks a CPU’s performance on certain SPU workloads and also provides reference figures to a real PlayStation 3. For those interested, the source may be found here;

5446 – Implemented cellMic library to allow for microphone emulation and currently supports 4 microphone types, namely, Standard, Singstar, Real Singstar and Rocksmith. See coverage in major improvements here;

ruipin

5781 – Implemented most sys_config syscalls which is necessary to boot and navigate through VSH in the emulator.

scribam

6078 – Replaced few hardcoded strings in the Vulkan Backend with corresponding macros from the Vulkan SDK.

Zangetsu38

6227 – Updated the thread scheduler to better support AMD Ryzen Zen+ and Zen2 architectures.

al3xtjames

6214 – Fixed an issue on Linux where the emulator would crash if a file named .directory was created by KDE dolphin in dev_hdd0/game/. This was resolved by ignoring file entries when adding game directories in the game list.

Yahfz

6193 – Set the default SPU Decoder to the LLVM recompiler and renamed the option from “experimental” to “fastest”.

Margen67

6182 – Fixed a typo in the settings dialog.

arabek

6169 – Fixed a build error when using cotire and Clang 8.0 branch.

Closing Words

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, MarioSonic2987, Whatcookie, JohnHolmesII and Digitaldude555.

RPCS3’s progress reports are written solely by volunteers and we’re looking for more dedicated writers to help us write them. If you have the knowledge, time and are willing to help, please apply here. And secondly, we have recently added functionality that makes it possible to unlock the framerate limit of many games. While this exciting new feature doesn’t work on every game, some big titles such as Uncharted 1-3, The Last of Us, Red Dead Redemption and more are able to go over their framerate cap of 30FPS for the first time! We require the help of the community to submit test results for as many games as possible to determine the effectiveness of the feature. For more details, please click here.

Now back to our regular coverage! This month saw a wide variety of important contributions from both lead developers, Nekotekina and kd-11 as well as many of our usual contributors. June also marked an important milestone in compatibility for the emulator as shown below. In this report, we’ll be going over the implementation of native MSAA in RPCS3 and mutli-threaded RSX workload support, as well as improvements to PPU scheduling and a significant overhaul to the DevOps environment.

In addition to the following report, further details of Nekotekina and kd-11’s work during June and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-06-10)
Status update from Nekotekina (2019-06-24)
Status update from kd-11 (2019-06-25)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

Although June was a slow month for compatibility improvements due to our testers taking a break, it did quietly mark the monumental milestone of the Playable category finally becoming the largest category in our compatibility list! This came to fruition from years of development, testing and re-working to ensure a perfect balance of accuracy and performance to emulate the PlayStation 3. While there is still much to be done, the momentum from these milestones continue to accelerate development. In addition to this, the Intro and Loadable categories decreased due to a few games being moved to Ingame or Playable for the first time. The Ingame category also saw a net decrease due to further maintenance done by merging duplicate game entries. This culminated into our second milestone, where the combined percentage of the Loadable and Nothing categories dropped below 1% for the very first time!

On Git statistics, there have been 24284 lines of code added and 14514 removed through 42 pull requests by 12 authors.

Major RPCS3 Improvements

Support for native MSAA (#6055, #6089)

For many months now, kd-11 has been systematically reworking the texture cache implementation in RPCS3 to behave much closer to the PlayStation 3. His work has made massive progress and fixed countless issues, all of which we have detailed in previous progress reports. This month, kd-11 finished the final piece of the puzzle, the feature that necessitated such a large re-implementation, Multisample anti-aliasing (MSAA). The PlayStation 3 had support for a few anti-aliasing techniques such as MSAA and CSAA for which support was previously not present in the emulator. Implementing these anti-aliasing techniques saw significant roadblocks early on due to the vast difference in implementation of these techniques between the PlayStation 3’s RSX and modern PC GPUs. As a stopgap measure, RPCS3 used primitive workarounds to notify games that MSAA was correctly working when in fact it was not. This led to worse image quality at native resolution (720p) on RPCS3 as compared to a PlayStation 3. However, with the introduction of resolution upscaling in 2017, the need for MSAA diminished as users were able to wipe out most aliasing by simply supersampling graphics i.e. setting the internal resolution to 4K and subsequently downsampling the image to the monitor’s resolution (1080p or 1440p). Throw in additional enhancements like anisotropic filtering and users were able to obtain graphics far exceeding the native hardware.

However, this was still a workaround and not the complete solution. Certain games (such as Red Dead Redemption) used MSAA on specific objects in the image (such as foliage) which simple supersampling could not replicate. After analysing the components that needed to be improved, kd-11 shared a broad roadmap to patreons detailing the work that was to be done to bring true MSAA support in RPCS3. Now that all items on that roadmap have been completed, kd-11 was confident that the roadblocks identified years ago were finally overcome and began work on implementing true MSAA support.

But nobody really expected this task to go ahead smoothly which became evident almost immediately. At the outset itself, kd-11 ran into quite a few issues. For starters, even with the basic implementation of MSAA, FPS of many games dropped to single digits. RPCS3 has come a long way since the seconds-per-frame era and kd-11 had no intention to take us back there. Debugging these issues, he identified significant hardware occupancy problems which were summarily addressed, bring performance back to normal. However, this was followed by even greater challenges as MSAA is one of the most heavily guarded techniques. Memory used for stencil buffering with multisampling is completely cut off from write access on NVIDIA GPUs. There are easy ways to tackle this on AMD GPUs, but all supported hardware needs to be taken into account to implement this feature successfully. While initialisation on NVIDIA GPUs is easily doable with CPU assistance, we are required to run a stencil test loop through all possible values which is simply too slow for RPCS3’s requirements. After discussion with other developers in the DXVK discord, kd-11 opted to iterate through all bits instead of all values for NVIDIA GPUs which reduced task complexity by a factor of 8. While this still left NVIDIA GPUs to potentially be 5 times slower than AMD GPUs in processing MSAA, it was thankfully enough to make MSAA easily usable on either GPU.

While the current implementation only supports a restricted version of MSAA that matches the PlayStation 3 layout, kd-11 plans to improve this further to allow more flexible options to be enabled to obtain visuals far exceeding the original hardware in every aspect. Many AAA games (such as Red Dead Redemption, Crysis series, Killzone series, Ratchet & Clank series etc.) rely on MSAA for proper visuals, and as you can see in the images below, visuals are leagues better with this feature enabled.

As a word of caution, please note that MSAA is now enabled by default in RPCS3 which increases the GPU utilisation in titles where this feature is used heavily. Anyone who has used RPCS3 before will know that previously, with most games there was no impact to performance when moving from native resolution to an internal resolution of even 4K on most modern GPUs. However, with the implementation of MSAA, some games may utilise MSAA x4 or even x8 and multiplying the internal resolution by 300% (to achieve 4K) may cause these effects to also multiply by the same proportion, thereby killing performance due to excessive GPU load, causing a bottleneck. Users are advised to test various levels of upscaling (150%, 200%, 300% and so on) to determine the maximum resolution that can be used without impacting performance.

Multi-threading support for RSX workloads (#6112)

Next on his plate, kd-11 implemented a wide set of changes improving various system types. First up is the major change of the lot being the introduction of multi-threading for RSX tasks. The previous set-up saw all RSX tasks occurring on the same thread as it was designed with older CPUs in mind (typically 4 core CPUs with SMT). However, with the current landscape of CPUs releasing with a minimum of 6 cores with SMT, it was time to take advantage of additional resources available. kd-11 analysed areas where the RSX workloads can be easily parallelised and the resulting performance benefits from the same. This led to the implementation of an additional off-loader thread to handle all PCIe transfers (from CPU to GPU) in parallel with the FIFO frame processing.

Overall, the performance uplift is quite significant (especially with the Vulkan renderer), but unfortunately most games may not see the full benefit of this improvement just yet as the performance bottleneck would just be moved to other components of the emulator such as the SPU. While the framerate may not increase much in some games, the working time spent emulating the RSX goes down significantly. In purely PPU + RSX workloads, the performance jump is between 20-50% depending on the system. Also, if users experience flickering visuals with Multi-threaded RSX option enabled, it is a sign that the extra thread is being frequently unscheduled by the operating system and is not able to keep up due to lack of CPU resources. Keeping low core count CPUs in mind, this option is disabled by default. However, more fine-tuning is planned to improve the benefits seen to such users as well.

Here’s an example of resource utilisation with RSX Multi-threading Off and On. While the improvement performance uplift is welcome, there is a decrease in Guest RSX utilisation with an increase to host metrics showcasing the shift of the bottleneck from the RSX to the other components.

But kd-11 didn’t stop there, during the research stage of MSAA integration he uncovered major performance differences between AMD and NVIDIA GPUs (especially on Windows). With some quick profiling by contributor Whatcookie, it was revealed that there was an excessive amount of sampler objects being created with Vulkan, and that this step was much slower on AMD drivers compared to NVIDIA drivers. Following this report, kd-11 reimplemented the vertex layout streaming under Vulkan to a streaming model which improves performance by avoiding a costly descriptor copy on every sub-block of a batched draw. This fixed issues on Radeon cards where performance would drop into seconds-per-frame territory with some games.

Afterwards, kd-11 uncovered more problems with how RPCS3 was using Radeon GPUs while integrating MSAA functionality. Thanks to the GPUOpen project and the Radeon GPU profiler, he quickly identified the source of the issue being shader occupancy on some passes where shaders were big, complicated and consumed too much register space unnecessarily. Once kd-11’s work on MSAA integration was merged, he continued investigating some generated shaders and optimizing them by hand, noticing that some shaders were consuming almost 50% more registers while being 3-10x longer in terms of instruction count than needed. This was actually causing shader load problems in some games where, even with a shader cache, it would take too long to preload the shaders. kd-11 restructured the GLSL emitter to output more optimal code and allow the SPIR-V generated to consume fewer registers (VGPRs) and improve CU occupancy.

Another issue identified (especially with the Vulkan renderer) was that there were simply too many barriers in place. Every imaging operation was converting the layout to the expected layout, then doing more barriers and subsequently reverting this change after. This isn’t a problem if done once or twice, but this was happening way too often and causing a performance bottleneck. The solution implemented was to reduce unnecessary memory barriers and resulted in a 30% improvement to Polaris GPUs. Both the above issues mainly affected performance in lower-end systems and weak GPUs, so users with powerful systems may not see the same level of benefits.

Finally, testers had pointed out a few more games with RSX bottlenecks and further digging revealed new bottlenecks in Zcull and index buffer uploads. Index buffer processing was accelerated by using SSSE3 which has the potential for up to 4x faster 32-bit uploads and 8x faster 16-bit uploads. The Zcull job dispatch for the Vulkan renderer was also rewritten to change the model from a one-entry-per-draw to a one-entry-per-commandbuffer system which is much more efficient in games that utilize Zcull heavily.

This led to a large increase in performance across multiple titles and especially many 60FPS games that were stuck in the mid-40s can now hit 60FPS if other parts of the emulator allow it which should improve playability, especially in fighting games. To keep the comparisons fair, we have disabled Multi-threaded RSX support to accurately showcase the performance uplift from the optimisations described above:

Restructure frame submission mechanism (#6069)

Finally, kd-11 restructured frame submission mechanism in the Vulkan renderer to avoid the NVIDIA driver deadlock bug that appeared since the 400 series drivers. Previously, RPCS3 used a fence signal and explicit waiting followed by manually submitting to the present engine with no semaphores. The idea was to control frame submission from the application instead of letting the driver do it, but this triggered a nasty deadlock with the drivers, sometimes forcing a hard reset of the system to get Windows systems working again. This has been resolved by just doing what every other application does and use signal semaphores in the driver instead. This path is bug-free, likely since it’s the most commonly used method by other vulkan applications. While this completely fixed the deadlock bug with NVIDIA GPUs, it also drastically improved frame-pacing in multiple titles, virtually wiping out micro-stuttering that plagued Yakuza games.

kd-11 also implemented a frame management system that allows resources to be released properly when the RPCS3 window is minimized, improving overall system performance and ability to multi-task. Finally, he finished off with a code clean-up which may slightly improve performance due to less overhead.

Implement TSX fallback (#6045, #6056)

You may or may not have heard but on May 15th 2019, Intel released new microcode updates to tackle a security vulnerability known as “MDS”. On Windows, users could optionally update to the newer microcode, while on Linux, most distros had the update available immediately. Users who received this update early may have noticed that RPCS3 was performing much slower. While it was easy to correlate the decrease in performance to the MDS mitigations, surprisingly they weren’t the real culprits for the loss.

The performance loss is actually the result of Intel resolving an unspecified erratum in their TSX implementation for Skylake+ CPUs. Intel doesn’t provide detailed release notes for microcode updates, so this information was actually gleaned through viewing old commit messages for the Linux kernel that were submitted by Intel employees. Luckily, it turns out that Intel added some new functionality that we can use to determine whether or not a specific CPU is impacted by this new microcode update. This new functionality is known as TSX_FORCE_ABORT. The presence of TSX_FORCE_ABORT indicates that a new MSR (model specific register) has been exposed with the ability to toggle On/Off 100% abort rate on TSX transactions. RPCS3 does not actually care for this feature, but every CPU that reports this feature can be considered to have a version of TSX which is known to behave differently.

Whatcookie, the new contributor to join our ranks, was responsible for researching and identifying the existence of TSX_FORCE_ABORT. Once this information came to light, he summarily implemented support in the emulator to detect the presence of this feature. If your CPU is affected, then you will now see TSX-FA appear in the second line of the log. In the latest version of RPCS3, many issues with TSX-FA have been resolved, but in the event you continue to experience issues, consider disabling TSX support in the Settings tab.

Once this detection was incorporated into RPCS3, Nekotekina aimed at improving the situation for users with the new microcode. To understand the purpose behind this change, we first need to understand a little about how TSX works. TSX allows us to write multi-threaded software that can operate without the need to use locks for synchronizing threads. After beginning a TSX transaction, any memory touched by that thread is marked as reserved. If another thread tries to touch that reserved memory, it will then cause an “abort” on the thread that reserved the memory, which means that it will roll back any changes made by the original thread, and then everything can continue on as normal. Previously, after aborting a transaction, the strategy was to simply try the TSX path again and again and again until it worked. However, with the latest microcode, the change in TSX implementation resulted in a massive increase in the abort rate, far beyond what is acceptable for RPCS3’s requirements thereby crippling performance. To alleviate this issue, Nekotekina implemented a fallback path where the emulator will now attempt to use the non-TSX path when it detects a high abort rate on the TSX path.

Our test case saw an improvement in performance from 30FPS up to 41FPS. This particular game saw 44FPS with the older microcode, and 39FPS with TSX disabled altogether. While the benefit of TSX has been reduced, it still offers users a performance uplift compared to the non-TSX path, which is always welcome! Nekotekina is also looking into further methods of improving the TSX path to bring performance back to previous levels. With that being said, in the event you experience a sudden decrease in performance and you see TSX-FA in your log, consider disabling TSX support for the time being.

DevOps Maintenance (#5178, #5964, #6046, #6058, #6071, #6142)

RPCS3 is a massive project totalling over 1100 files and 70 folders (excluding submodules), and as such, it can be quite messy in places. While many volunteers and core members have spent significant time working on keeping the codebase clean, technical debt and other various cruft inevitably creeps in. In June, there was a spike in pull requests dedicated to cleaning up the project itself. While these changes have limited effects on game compatibility and performance, they do have a strong effect on keeping the project open to new contributors, as well as prevent new bugs from being introduced.

Compiler warnings – JohnHolmesII & MSuih

Anyone who has programmed before is certainly familiar with compiler warnings. For those who have not, here’s a brief overview: a compiler will review written code before compiling it, and during this review, it might emit warnings about things that it sees as potentially incorrect code, misleading code, or even stylistically poor code. Most projects are very particular about the way that warnings from the compiler are handled, with some even going so far as to refuse to accept any code that creates warnings. RPCS3, unfortunately, has been on the opposite end of the spectrum in this department.

RPCS3 did not have any additional warnings enabled besides the ones that are enabled by default in the compilers, which is not the best practice to follow. What’s worse is that despite not having many warnings enabled, thousands of warnings were still emitted at compile time. The automated Travis build system, for example, had logs stretching over 8000 lines, most of which were warnings. And if all of that was not enough, the warnings emitted across the 3 compilers used (GCC, Clang, and MSVC) were all different! MSVC being different is understandable, but GCC and Clang have very compatible warning info, and should have been very similar.

The situation was improved by two major contributions. The first, by JohnHolmesII, focused on the Unix side of things while the second, by MSuih, was a follow-up with a focus on cleaning up the warnings in MSVC. The CMake build system was slightly refactored, such that GCC and Clang are now treated as very similar compilers, with MSVC being the odd man out. The warning flag -Wall, a staple in just about any project and usually considered a minimum, was only now enabled for the Unix compilers, as a baseline. It turns out most of those thousands of warnings were trivial fixes, such as adjusting some data types. Some were a little sinister, such as a hidden truncation. Part of the refactor of CMake also included intentionally hiding some warnings which was done for two major reasons. The first being that not all warnings are good warnings, and the second being that fixing many of the existing warnings required significant effort from core developers. With this, the compiler warnings clean-up was a success, bringing the Travis log from over 8000 lines to around 1000 lines, with most lines now being normal compiler output. RPCS3 still has a ways to go in terms of warnings compliance, but significant headway has been made. An important side effect of cleaning up the warnings is that any new warnings introduced will stand out significantly, preventing them from ending up in the code base. Warning systems can be relatively self-maintaining, if they are enforced properly.

Clang-tidy, warnings and modernization – scribam

Finally, we’ll wrap up the major improvements section with the work done by contributor scribam, who went on an absolute tear of work into the nitty-gritty. First, he applied -Wpedantic to the codebase, and fixed a number of the warnings generated there. This warning flag is still not enabled by default, but this was a good start to getting there eventually. Then, scribam ran the linting utility clang-tidy over the codebase. A linting utility is a tool that looks for various issues in code, including everything from things that look like warnings to style recommendations. Importantly, tools like clang-tidy find out-of-date functions and coding techniques that may be harmful to the codebase. An example is the find_first_of and find_last_of string search functions built into the C++ standard library. Clang-tidy was able to remark that searching for single characters instead of entire substrings is substantially faster, allowing some areas of the RSX code that use these functions for shader parsing to become ever-so-slightly more efficient. Quite a number of readability and modernization related suggestions from clang-tidy were implemented, improving the maintainability of the codebase. His final contribution was to update the required GCC version to 8, as the modernization features require this.

DevOps can be nasty work sometimes, as it can seem like a great deal of effort for seemingly no tangible reward. Much of the updating of the codebase requires parsing hundreds of other people’s code, and despite helpful messages from linting tools, the proper changes are not always obvious. Still, this kind of work is necessary for the longevity of a project on this scale, and it is a never-ending task. There is still much more to be done in the short term, including a good deal more work on cleaning warnings, and working on updating the build environment for Windows to be as up to date as possible.

Games

Haze

To kick off the games showcase for this month, we have Ubisoft’s first-person shooter Haze. This PlayStation 3 exclusive title previously suffered from graphical issues such as missing text and a permanent “yellow filter” while ingame. All these issues have now been fixed, allowing this title to be fully playable on RPCS3.

Backbreaker Vengeance

The successor to the infamous arcade american football game, Backbreaker Vengeance, finally became playable after the last remaining graphical issues were fixed by kd-11.

Kidou Senshi Gundam UC

Another Gundam title finds its way to playable with the last remaining bugs in Kidou Senshi Gundam UC being addressed. While this game had good performance and graphics, it would previously crash during certain custom cast scenarios. Thanks to the accuracy improvements with the emulator, all such issues have now been summarily fixed.

2010 FIFA World Cup: South Africa

One of the few FIFA titles to not arrive on PC, 2010 FIFA World Cup, finally became playable this month. Users who missed out on the World Cup can now enjoy this game on PC with RPCS3.

Fight Night Champion

The console exclusive boxing title, Fight Night Champion, now goes ingame on RPCS3. However, this game seems to suffer from low performance during the matches keeping it from being playable.

SEGA Rally Online Arcade

If anyone was looking forward to racing titles, look no further. This month SEGA Rally Online Arcade was found to safely go ingame. While the game seems to have good graphics, it suffers from low performance keeping it from being playable.

DJ Hero

To finish off the list, this month saw DJ Hero progress past the title screen and go ingame! While the DJ turntables are still not supported on RPCS3, connecting a controller as a Guitar pad allows users to play the DJ-Guitar remix stages. Do note that users may be required to use cheats to unlock all songs and then play the guitar levels.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

6047 – Implemented std::bit_cast and utils::popcnt32, and also addressed certain warnings from the GCC compiler;

6050 – Fixed regression caused by the above pull request;

6052 – Fixed regressions in cellVdec affecting titles such as Project Diva F and Gran Turismo 5 caused by PR 6047;

6060 – Changed PPU thread behaviour to yield on disk access operations;

6085 – Added a workaround in sys_rsx_context_iomap to prevent failure with CELL_EINVAL on vm::main;

6056 – Implemented a fallback for TSX codepath in certain scenarios. See coverage in major improvements here;

6108 – Fixed regression affecting SPU loop detection after the above pull request;

6128 – Fixed file encoding in vm.cpp to UTF-8 BOM after PR 5178.

kd-11

6069 – Implemented frame management when the game window is minimized, allowing resources to be released properly; Removed unnecessary code from the window classes; Removed frame submission control from the application; Fixed the Vulkan fullscreen crash. See coverage in major improvements here;

6055 – Added MSAA support. See coverage in major improvements here;

6089 – Set default MSAA level to auto in settings. See coverage in major improvements here;

6112 – Introduction of Multi-threading support for RSX workloads and various other optimisations. See coverage in major improvements here.

elad335

6068 – Used std::atomic_thread_fence instead of _ReadWriteBarrier() with MSVC as the latter is deprecated;

6099 – Fixed undefined behaviour by trying to obey strict type aliasing rule in ppu_ref cache access;

6120 – Fixed a regression caused by the above pull request;

6115 – Fixed potential overflow in sys_vm;

6116 – Fixed Big-endian endianess arch support in semaphore_406e;
6143 – Prefer vm::ptr<>::ptr over vm::get_addr; Prefer vm::_ptr/base over vm::g_base_addr with offset; Added methods atomic_t<>::bts and atomic_t<>::btr; Removed obsolete rsx::thread::Read/WriteIO32 methods; Removed wrong check in semaphore_release; Added handling for PUTRx commands for RawSPU MFC proxy; Prefer overloaded methods of v128 instead of _mm_… in VPKSHUS ppu interpreter precise; Fixed more potential overflows that may result in wrong behaviour; Added io/size alignment check for sys_rsx_context_iounmap; Added rsx::constants::local_mem_base which represents RSX local memory base address; Removed obsolete rsx::thread::main_mem_addr/ioSize/ioAddress members;

6146 – Use vm::get_addr instead of manually subtracting vm::base(0) from pointer in texture cache code; Prefer std::atomic_thread_fence over _mm_?fence(), adjust usage to be more correct; Used sequentially consistent ordering in semaphore_release for TSX path; Improved memory ordering for sys_rsx_context_iounmap/map; Fixed sync bugs in HLE gcm; Use release memory barrier in lwsync for PPU LLVM;

6153 – Fixed regressions caused by the above pull request.

scribam

6046 – Cleanup code in few areas and addressed few compiler warnings. See coverage in major improvements here;

6061 – Optimised TextGlyphs used in RSX debug overlay;

6048 – Updated AppVeyor and Travis to use the newer Vulkan SDK; Updated VulkanMemoryAllocator to the latest tagged version; Updated stb_image.h and stb_truetype.h files;

6071 – Set minimum supported version of GCC to 8.1 and updated the documentation accordingly. Also, moved building instructions to dedicated BUILDING.md file with additional explanation on how to install GCC on Ubuntu systems. See coverage in major improvements here;

6065 – Added constexpr where necessary within RSX code;

6058 – Applied some Clang-Tidy fixes to the codebase. See coverage in major improvements here;

6017 – Added FUNDING.yml that displays the “Sponsor” button at the top of the RPCS3 repository;

6092 – Removed unused variables.

Megamouse

6057 – Fixes an oversight in PR 4450 that added a slash to the game paths in games.yml whenever a game was booted. This bug led to funny paths such as: “C:/games/BLUS12345/PS3_GAME/USRDIR////////////////////////////////EBOOT.BIN”;

6081 – Fixed an issue with the Linux evdev pad handler that caused the KEY_BACK and KEY_HOMEPAGE buttons used by some gamepads to not register in-game. Usually these key IDs are used only by keyboards, but in some cases they are also chosen because the regular button IDs don’t represent the actual button as accurately as these key IDs. It’s a mystery how this bug wasn’t observed in the original implementation since it was added to fix exactly this issue. Currently only those two keys are supported. New keys will have to be added incrementally if the need arises;

6031 – Fixed in-game pad interception issues such as a bug that kept the pad input intercepted after a game aborted a message dialog while using the Qt dialogs instead of native UI dialogs. This caused an inability to control the game any further.
Native dialogs now also skip the on_close callback since it is not called in this function on an actual PlayStation 3. Also, fixed a bug that caused a subset of games to still receive input while native UI dialogs were open;

6095 – Added the option “Remove All Caches” to the game context menu. This removes all SPU, PPU and shader caches for the selected game;

6105 – Properly activates GUI buttons when rebinding your game pad with Linux evdev device handler. Previously, when using the evdev device handler, the buttons in the pad settings dialog were initially shown as disabled until you pressed a button;

6109 – Based on user feedback, increased the maximum mouse acceleration from 5 to 10;

6122 – Improved file decryption where previously users were only able to decrypt .sprx files using the GUI. Now users can decrypt .bin, .self, all binary files (.bin, .self, .sprx) and all other files. Also, the file endings of the decrypted files will be set to .elf in case of .bin or .self.

Whatcookie

6045 – Added detection for new intel microcode releases that are known to change the behavior of TSX. See coverage in major improvements here;

6096 – Allows linux to sleep in more situations when sys_timer_usleep is called and improved accuracy of usleep.

MSuih

6138 – Added a timeout for vulkan device enumeration and adds a proper exit strategy.

ruipin

6097 – Fixed dynamic_library::loaded.

JohnHolmesII

5964 – Cleanup compiler warnings present with the project. See coverage in major improvements here;

kukkamario

5178 – Refactored header files and code cleanup. See coverage in major improvements here;

RainbowCookie32

6063 – Updated DualShock 3 message in Pads settings.

Talkashie

6067 – Corrected a typo.

Upcoming

As we mentioned at the start of this report, developer eladash has been working on ways to push performance in some games beyond their framerate limit in a generic way that doesn’t require per-game patches. And when we say the framerate limit, we’re not referring to only 30FPS. Some games that run natively at 60FPS on the PlayStation 3 can be pushed further as well, bringing the performance well past a 100FPS! To reiterate, if you wish to help the developers efforts of identifying games where this is possible, please click here.

Closing Words

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, Asinine, JohnHolmesII, Whatcookie, Digitaldude555, Megamouse and MarioSonic2987.

This month saw some major leaps by Nekotekina and kd-11 on the SPU and RSX fronts. Nekotekina implemented SPU PIC support while kd-11 improved the surface cache implementation. Meanwhile, Megamouse made multiple improvements to the UI, GalCiv overhauled the DualShock 3 pad handler and ruipin tackled regressions in the SPU LLVM backend when using Mega SPU block size.

In addition to the following report, further details of Nekotekina and kd-11’s work during May and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-05-09)
Status update from Nekotekina (2019-05-14)
Status update from kd-11 (2019-05-27)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

This month RPCS3 reached one of the most significant milestones in game compatibility. After intensive testing and merging of duplicates, as mentioned in the previous month’s report, the Playable category has tied with the Ingame category at 43.71%. RPCS3 has surely come a long way to have the Playable category stand on the cusp of overtaking the Ingame category. On the other hand, Intro section saw a modest drop while the number of Loadable and Nothing games remained unchanged. For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

On Git statistics, there have been 9391 lines of code added and 5430 removed through 45 pull requests by 12 authors.

Major RPCS3 Improvements

Surface cache improvements (#5937)

At the beginning of May, kd-11 began improving how RPCS3 handles the surface cache, which is not trivial since the way the PlayStation 3 renders to a framebuffer is different from a PC. Originally kd-11’s plan was to implement variable-sized framebuffers to solve a problem in the rendering method which the emulator used. Previously, RPCS3 kept a list of all the available framebuffers and their locations in the systems memory and while this was fast, it had its limitations. To keep try and keep this as simplified as possible, let’s take a look at a few diagrams on how RPCS3 handles framebuffers:

Variable-sized framebuffers was an idea where instead of making a viewport of the small red area you see above before drawing the circle inside, we would create a big area that covers the full framebuffer (the entire gray area) but only render to the red area. This would make the framebuffer’s size “variable” as it is allowed to be larger than the exact area being rendered to and adjust itself dynamically to preserve data. This sounds like a good idea, but there is one critical flaw and that is memory ordering.

Every drawcall in RPCS3 tags the framebuffer with a value to identify how new it is, so that when we recreate an image it always starts drawing the oldest pixels first. In the example above, It doesn’t matter if we render the red or blue circle first since they are separate from each other. Drawing the black triangle is ok. Drawing the blue circle is also fine. But, when it’s time to draw the red circle, the grey area is inherited with the black triangle now and is tagged as part of the newest data. This results in the blue area being older than the grey area resulting in it being completely overwritten when we draw the scene using variable-sized framebuffers.

Thus, kd-11 had to go back to the drawing board and come up with a solution. Which was to perform intersections for all reads and writes. Instead of the red area extending all the way to the boundary of the frame buffer (gray area), we can now do this:

In the above example, we’ve now split the framebuffer into three surfaces. And the red area is replaced by a new framebuffer and tagged as new which allows us to update it separately from the rest of the surfaces.

This fixed a variety of issues with games having a black screen in certain scenarios. In Ratchet & Clank: A Crack in Time, this happened at certain spots throughout each level making it hard to progress. With Resistance 3, it occured when HP was low. And finally, Nascar 14 and Nascar Inside Line was also outputting a black screen whenever you crashed into a car for a few seconds. NeverDead’s graphics were also fixed, removing the split-screen effect. There’s sure to be many other games that improved from this, so feel free to let us know if you find anything!

PIC support for SPU (#5944)

Continuing to optimize emulation of the Synergistic Processing Unit (SPU), this month Nekotekina has added support for Position-independent code (PIC), bringing with it less stuttering and more frames per second!

To recap, the SPU recompiler works by taking in chunks of game code meant to be run on the PlayStation 3, analyzing it and generating code that can run on your computer (with the use of LLVM), which is then stored as a program. Programs are created every time a unique chunk is identified and cached to be re-used when the same chunk is executed again at a later point. This helps us avoid the arduous process of recompilation as emulation is momentarily paused every time a new game chunk is recompiled.

Since the emulator has no visibility into the game code, there is no way to speculate which chunk will be run and consequently, all such recompilation is done at runtime. This causes a dip in performance when new SPU programs are recompiled. Thankfully, this process generally only lasted for a few minutes and performance uplift obtained from using the SPU LLVM recompiler was well worth the wait. However, for certain titles (famously Red Dead Redemption), the SPU recompiler would continuously compile new SPU programs (colloquially referred to as infinite compilation) giving the impression that the game was continuously generating unique chunks of code. However, on closer inspection, this was not the case. While the games used the same chunk of code, it was loaded into a different memory address each time. In the erstwhile implementation, the SPU programs used absolute addressing, making them reliant on being loaded into a specific location. If the same chunk was loaded in any other location, it was wrongly identified as unique code and recompiled once again.

This is where Nekotekina’s pull request comes in, allowing generation of these programs in a position-independent way, making them loadable anywhere and thus reusable. This completely solved the issue of infinite compilation and greatly reduced the number of SPU programs compiled. For example, in the case of Spider-Man Shattered Dimensions, just thirty seconds of gameplay would previously generate over 11,000 SPU programs with only 2,054 of those programs actually being unique. Now, with Nekotekina’s improvements, this title generates only 2,113 SPU programs in the same amount of time.

This is a huge improvement that benefits almost every game when using the SPU LLVM recompiler. For example, where the FPS used to drop to 11 when using an intensive attack in Fist of the North Star: Ken’s Rage, it now stays at a smooth 25-30FPS. Currently this only works with either Safe or Mega SPU block sizes.

Comparison of how many SPU modules are compiled before and after these changes across various games. Note that some games shown in this graph (GTA V or One Piece) previously were compiling SPU modules indefinitely.

Improved Game Collections Support (#4450)

Among the various GUI improvements this month, Megamouse also reworked the way RPCS3 handles game collection discs. Before we dive in to the changes made, here’s a quick overview of how PlayStation 3 discs are structured. Every disc contains 2 folders, namely, PS3_GAME, PS3_Update and one file, PS3_DISC.SFB. This is the general layout for the game discs. The PS3_DISC.SFB file does not contain any relevant data within it other than the product ID. However, while the file does not contain any critical information, the file itself is used by RPCS3 to identify the existence of a disc-based game. Once identified, the emulator scans the PS3_GAME folder to pull all the necessary details about the game and populates the gamelist to allow users to easily launch the game.

With game collections, there were two distinct file layouts. The first (and more common) layout followed the above layout with all games being stored within the PS3_GAME folder and appearing as a single game in the game list. Once launched, users will be prompted with the choice to select which specific game they wish to launch and the selected game will boot in a new window.

Launch screen of Kingdom Hearts HD 1.5 ReMIX collection that allows selection of titles present within the collection

However, with the second type, the file layout is a little different. Here, each game in the collection is stored separately and hence you find additional folders such as PS3_GM01 and PS3_GM02 within a disc. This can be seen in discs such as The Disgaea Triple Play Collection, Ultimate Stealth Triple Pack and Ultimate Action Triple Pack among others. When inserting these discs into an actual PlayStation 3, three separate disc games would appear in the game list as opposed to just one. In RPCS3’s previous implementation, only the contents present in the PS3_GAME folder were scanned and populated to the gamelist and hence the games present in the additional folders were ignored.

Here’s a comparison showing the file layout difference between regular discs and the special collection discs

To fix this, Megamouse reworked the existing implementation of detecting game files by adding m_game_dir, a new variable to point the relevant game folders to. Now, instead of hardcoding the PS3_GAME folder, the emulator will search for other folders and identify whether they are also games and correctly populate them into the RPCS3’s gamelist.

Batch processing for all titles (#5519)

Lastly, we have a quality of life improvement by Megamouse, who added the option to compile the PPU cache for all titles at once, but also to clear all caches or custom configurations. Gone are the days of having to compile the PPU cache for each title individually. While this improvement may not be as exciting as the ones mentioned above, it does a much needed comfort that regular RPCS3 users will definitely come to appreciate.
This new feature can be accessed under File > All Titles.

Games

Yakuza 3 & 4

Thanks to improvements made by kd-11 to frame data handling, Yakuza 3 & 4 no longer have exploding vertices when using AMD GPUs. This change brings further parity of graphical output between Nvidia and AMD GPUs to provide users a more uniform experience irrespective of the hardware used. Check out the clip below to see the improvement in action:

Genji: Days of the Blade

Genji: Days of the Blade is now the second PlayStation 3 launch title to become playable after Ridge Racer 7.

Cash Gun Chaos DLX

From the arcade corner, this month saw Cash Gun Chaos DLX become fully playable. This arcade style shooter works with steady performance and graphics. Check out some gameplay footage below:

SIREN: Blood Curse

The PlayStation 3 exclusive survival horror game, SIREN: Blood Curse was found to be playable this month. A user from our forums, jade010, managed to finish this game on RPCS3 as well. Please note that you need a DualShock 3 or 4 to play this game as it makes use of the controllers gyroscope.

One Piece: Pirate Warriors

This is one of the titles that has improved after Nekotekina’s PIC support for SPU. While the game runs well with great performance and graphics, it suffers from audio stuttering when there are too many enemies on screen, keeping the title from being playable.

Mortal Kombat vs. DC Universe

This crossover fighting game is now playable with great graphics and performance. Previously, this title suffered from various graphical glitches and did not work with the PPU LLVM recompiler. However, thanks to the improvements made to the emulator in the recent months, all these issues have now been summarily addressed.

Cross Edge

Another niche console exclusive, Cross Edge, a tactical RPG title that features characters from a wide variety of franchises, was found to be playable this month.

Dark Mist

To finish our roundup of games, Dark Mist, was found to be playable this month. This title previously required LLE libvdec to be enabled which reduced performance enough to keep it from being playable. However, with improvements made to the emulator, the same is not required anymore. Users should note that this title currently only works with the OpenGL renderer.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

5901 – Fixed few regressions in SPU LLVM caused by PR 5882;

5915 – Fixed a regression in SPU LLVM where Drakengard 3 couldn’t create a new savegame;

5923 – Improved SPU Analyser when block size is set to Giga;

5975 – Fixed regressions caused by the above pull request;

5944 – Provided PIC support for the SPU LLVM recompiler. See coverage in major improvements here;

5967 – Fixed regressions caused by the above pull request;

5976 – Fixed a minor bug when using Safe SPU block size. Also, fixed broken BISLED instruction behaviour in SPU ASMJIT recompiler;

5993 – Improved performance when using TSX instructions;

6020 – Fixed regressions caused by the above pull request.

kd-11

5895 – Removed a workaround for AMD GPUs that is no longer needed with newer drivers (since version 19.4.3), fixed texture cache search for flip source, fixed confirmed_range calculation for hardware blit transfers (software memory write barrier for flushing), fixed out-of-bounds transfers due to no bounds check when testing local resources for overlap and fixed clear failure due to null stencil mask triggering an assert;

5922 – Allowed certain drivers to bypass the window state polling if they properly handle OUT_OF_DATE and/or SUBOPTIMAL return codes to signal that the window surface has changed. This offsets the surprisingly large penalty of polling the window size from Qt on some linux window managers. Also, fixed a typo in OpenGL renderer code;

5937 – Improvements to the surface cache. See coverage in major improvements here;

5977 – Transition attachments to LAYOUT_GENERAL in case of a feedback loop. Fixed appearance of garbage along polygon edges in some post-processing passes. Also reversed this transition when rendering goes back to normal. Optimized transitions from LAYOUT_GENERAL to some common read/write layouts;

5979 – Fixed a bug where vk::sampler::matches would always return false because the info was never initialized. Added the first step of centralized resource management by managing a sampler pool instead of pushing and deleting resources every frame. Fixed performance gap between AMD and Nvidia GPUs found in Ninja Gaiden Sigma;

5988 – Bumped max allocated draw call resources up to 16K from 4K to avoid easily running out of resources mid-frame in heavy scenes. Used a simple FIFO queue for frame data handling. In the future, this can easily be expanded to use a present scheduler thread for frame-pacing support; Fixed exploding vertices on Ratchet & Clank and Yakuza titles when using AMD GPUs; Fixed a regression caused by PR 5565 where inFamous crashes during intro;

5995 – Fixed staging buffer size calculation where pitch was row-aligned to 1 byte instead of 4 bytes; Fixed gsl::span assert when using OpenGL renderer in some games; Fixed Ni no Kuni stuttering ingame caused by the above pull request; Fixed a crash in Uncharted: Drake’s Fortune when the Debug output option is enabled;

6011 – Ensured the current renderpass matched the image properties even when a cyclic reference was detected; Solved problems due to mismatching layouts and renderpasses;

6025 – Refactored out framebuffers from the renderer core, bumped up shader cache version and implemented a proper cache with sorted queues for faster searching;

6030 – Updated glslang;

6039 – Fixed aux context usage when handling swap queues. The aux context did not have its own descriptor pool and borrows from the laggy frame context which had a pool that was still in use; Refactored out GLSLCommon from VKHelpers since VKHelpers was included in GUI code to assist with context initialization. This removed a lot of compiler warning spam about unused static functions declared in the header.

Megamouse

5519 – Fixed wrong module count in Qt Compilation Dialogs; Fixed a glitch that stacks Qt Compilation Dialogs; Don’t add games to the Recent Games List when the add_only option is used; Ignore operations other than boot when rebooting the last game from the GUI; Show all unique game data entries instead of only the first; Added option to batch create PPU cache and batch remove PPU cache, SPU cache, shader cache and custom configurations;

5916 – Properly scale game icons as well as the compatibility circle-marks in both the game list and the game grid on high-DPI screens. This is especially noticeable on 4K screens where, prior to this pull request, the icons would look jagged and aliased;

5655 – Updated the Travis Mac build process to Xcode 10.2 and removed an unnecessary workaround for Xcode 10.1;

5929 – Added per controller color picker in the pad settings dialog for DualShock 4 controller LEDs and future implementations of other game pad LEDs;

5960 – Added an optional custom pad config for every game.
The configs will be saved per game in config/custom_input_configs/. The configs behave exactly like the normal pad configs just with their own config files including the profiles. New icons were added to show which game has its own pad configuration. Users can assign a new config from the game list using the context menu;

5980 – Added 10 step navigation to the native in-game save data dialogs. You can now use the L1 and R1 buttons to jump 10 entries back and forth in the list, which makes navigating through big lists faster and more responsive;

5978 – Added simple multipliers for left and right analog sticks to the input config files. The allowed range is 0-200 (0.0 – 2.0) with the default value being 100 (1.0). Access to this functionality through the GUI is also planned to be added; Fixed a crash in the pad settings dialog that was caused when clicking “Filter Noise” while the current pad was disabled. The button will now only be activated when there is a pad connected; Fixed a bug that shows the wrong color when selecting LED colors; Fixed a bug that resets the LED colors when clicking the vibration checkboxes; Fixed a crash when downloading the compatibility database twice in a row; Fixed a crash when trying to load an empty PARAM.SFO in the save data manager; Removed unnecessary warnings;

4450 – Improved support for game collections. See coverage in major improvements here.

scribam

5949 – Fixed a minor typo in OpenGL and Vulkan renderers used only for debugging;

5956 – Removed redundant semicolons. Added missing #pragma once directive;

6042 – Removed duplicated condition in pipeline_props struct equal operator in Vulkan.

MSuih

5905 – Used RPCS3’s wiki for DualShock 3 and DualShock 4 troubleshooting instructions;

5982 – Added a Max SPURS threads option in the Debug tab;

6040 – Removed the SPU verification setting from Debug tab and changed the Log shader programs setting’s behaviour.

Whatcookie

5919 – Doubled dpad repeat rate;

5998 – Fixed inaccurate FPS counter in the performance overlay on Windows when the detail level is set to High;

6006 – Enabled window resizing again on Linux with amdvlk/amdgpu-pro drivers.

elad335

5907 – Fixed sys_rwlock_runlock on waiting readers and sys_rwlock_wlock timedout event; Returned ESRCH if PPU thread ID was not found in sys_cond_signal_to;

5939 – Fixed 3D swizzled texture to linear conversion. This fixed an old regression in Call of Duty titles.

drysalter

5899 – Fixed minor visual bugs in RPCS3’s GUI;

5932 – Further improvements to the Skyline and Envy themes.

GalCiv (RipleyTom)

5933 – Improved the DS3 pad handler on Windows. Now supports Sony’s official driver distributed with the PlayStation Now installer.

ruipin

5951 – Fixed several regressions caused by PR 5923 when using the SPU LLVM recompiler with Mega SPU block size.

z0z0z

5963 – Used setenv instead of qputenv for QT_AUTO_SCREEN_SCALE_FACTOR parameter.

Exfiltratior

5930 – Fixed cache deletion on Linux.

Closing Words

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by Asinine, elEnemigo, MarioSonic2987, HerrHulaHoop, Digitaldude555 and Megamouse.

This has been a very busy month, which saw many contributions from our regular developers and even a few newcomers. Major improvements have been made to RSX emulation by kd-11, fixing the texture cache and improving on the shader decompiler. Meanwhile, eladash in his usual style, fixed a multitude of bugs relating to savedata handling and the PPU/SPU interpreters/recompilers. Numan (Inviuz) implemented a fringe syscall needed for Metal Gear Solid 4 to boot and Nekotekina squeezed quite a bit more performance out of the SPU LLVM path. To also improve the visual aspect of the emulator drysalter created two beautiful new themes and lastly GalCiv expanded DualShock 3 Support to Linux. These and many more improvements have all contributed to making RPCS3 both a better piece of software and a better emulator, moving a bunch of new games into the Playable category.

In addition to the following report, further details of Nekotekina and kd-11’s work during April and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-04-07)
Status update from kd-11 (2019-04-23)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

This month RPCS3 reached another milestone in game compatibility. It’s the first time that the Playable category surpasses 40%! As a result, the Ingame and Intro categories see an equivalent, while the Loadable category goes down to 29 games. We are also gearing up to undertake further maintenance to the compatibility list by identifying and merging duplicate entries in the coming months! For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

On Git statistics, there have been 9921 lines of code added and 2967 removed through 113 commits by 16 authors.

Major RPCS3 Improvements

Texture cache fixes and shader decompiler improvements (#5785, #5813, #5860)

In April, kd-11 switched focus to tackle the blit engine portion of the texture cache. Even with the initial implementation, multiple games responded positively to the improved accuracy of the texture and surface cache in general without too great of a performance hit. kd-11 added a few more improvements along with those patches, such as implementing a few missing compressed formats, working on finally getting bit-casting of texture data working across a DEPTH_STENCIL<->COLOR border.

For those interested in technical details, this is particularly important since the RSX has no concept of texture storage (this is usually just an API convenience for user-side), and arbitrary recasting of memory pools is quite common. For example, you can store a normal 32-bit RGBA texture somewhere, even render to it and then for the next pass cast it as a 2×16-bit format like RG_16 or RG_16FLOAT, or even a depth format like DEPTH24_STENCIL8. Since many PlayStation 3 formats do not map properly onto modern PC graphic cards, it is essential to implement a proper decoding step to correctly emulate such formats. kd-11 has set this up with compute for Vulkan, while OpenGL has the benefit of a flexible pack/unpack configuration for memory copies between textures and arbitrary buffers. These improvements have fixed effects such as broken lighting, depth of field and fog in titles such as Uncharted 2.

Also included in this changeset is an improvement to RPCS3’s Vulkan memory management which should reduce the likelihood of encountering a Working buffer not big enough crash.

While testing the texture cache improvements, kd-11 picked up a side task to improve the shader decompiler. There were several examples of titles where different hardware was producing wildly varying results. This includes titles like Ridge Racer 7, which was playable on AMD GPUs, but a flickering mess with NVIDIA GPUs. A common issue with graphics hardware is that they are often fundamentally very different internally, something the manufacturers can often get away with due to the API-only access provided to programmers and end users. This means that, unlike a CPU, the same inputs to a function may not always provide the same results on different hardware.

In this case, there was a significant arithmetic drift in NVIDIA GPUs where some mathematical operations seem to have been operating at seemingly low enough precision that a comparison test with a constant provided by the game was failing. AMD and Intel hardware was seemingly unaffected, so kd-11 started investigating further and found many similar problems.

To handle these variances, kd-11 spent a whole week throwing random instructions with corner-case inputs at PlayStation 3 to better document the behaviour of the RSX shader processing unit. This has yielded increased accuracy in several areas where RPCS3 was incorrectly rendering many graphics and hiding few hard-to-fix bugs.

To round off coverage on kd-11’s work this month, here’s comparison images of Ridge Racer 7 now displaying foliage on Nvidia GPUs and Battlefield: Bad Company no longer has glowing lights on screen:

kd-11 had also set up a framework to take advantage of graphic cards offering native FP16 support to both improve accuracy of emulation as well as to potentially improve performance in the future, especially with newer GPUs supporting this feature out of the box e.g. Turing and Vega.

eladash’s improvements (#5749, #5792, #5850)

As with last month, eladash fixed so many bugs that it’s nigh impossible to cover them all adequately, but here’s the summary.

One of eladash’s focuses was to fix issues with how RPCS3 handled savedata. This was apparent in some games such as Lord of the Rings: Conquest, where it would try to read the savefile directory even though it didn’t exist yet. On a real PlayStation 3, an error code is returned to the game, telling it that the directory doesn’t exist. However, before this fix, the emulator never threw this particular error code, so the game tries to load a nonexistent savefile, leading to an access violation crash and rendering the game unplayable unless you already have a savefile in the directory. After eladash implemented the proper handling of this situation, Lord of the Rings: Conquest and Fritz Chess were fixed, the latter going from the Intro category straight to Playable!

eladash also tackled a tricky race condition that was plaguing RPCS3. Race conditions arise with complex multi-threaded applications like RPCS3 where the timings between different parts of the program that run concurrently are critical to the proper functioning of the program. This can be caused by two threads depending on the same data, or expecting that data to be in a specific state when they start working on it, yet that state might be altered if unexpectedly the other thread operated on it first. These issues are difficult to debug and result in issues such as the infinite loading screen in Family Guy: Back To The Multiverse and the inability to load savegames in Cars: Race-O-Rama. eladash fixed both of these issues by simply making the thread sleep (wait) in some places, which corrected the timings.

eladash’s next focus was on improving the PPU and SPU interpreters and recompilers. This includes fixing incorrectly handled instructions, getting all these different decoders to behave more similarly, yielding small performance improvements in certain cases. This fixed the orb rendering in SoulCalibur IV’s menu screen with the PPU Precise Interpreter, blackscreen and crashes in Sonic Unleashed and makes the PPU LLVM Recompiler behave more accurately and closer to the PPU Interpreter.

Finally, let’s look into how he corrected the handling of SIMD with saturation instructions on both the PPU Fast and Precise decoders. When decoding these SIMD instructions we iterate over all the elements individually, apply the instruction and write the results in the appropriate element of the destination register. That last part however causes issues when the destination register is also one of the source registers, as we are now overwriting the source register, corrupting data for the next element. The solution was simple: Just use a temporary variable as a destination and copy that into the actual destination register at the very end. This specifically fixes the loading icon in Beyond Two Souls.

Initial sys_overlay support (#4007)

Many of you probably have heard of Numan’s (Inviuz) implementation of the infamous sys_overlay syscall, used (as far as we know) exclusively by Metal Gear Solid 4. Ever since Numan opened this pull request at the end of 2017 everyone in the community has been eagerly waiting for it to be merged and now it has finally happened!

While it has been talked about quite often, and also teased in this progress report, here is still a quick recap for the uninitiated:
Syscalls are functions, provided by the PlayStation 3’s OS, acting as a way for programs to interact with the system. There are many syscalls for everything from allocating memory to communicating with USB devices, and while RPCS3 implements many of them, some are still missing. One of those missing syscalls was sys_overlay, which was not implemented till now due to having virtually no documentation on its functionality as well as only a single game making use of it. This syscall provides games with a special way to load external self files and is used by Metal Gear Solid 4 to boot a PS1 arcade stage.

Now that it is implemented, we are one step closer to getting MGS4 to boot. Unfortunately, there is still much to do until that happens, but there are fixes in the works from our developers, so stay tuned!

SPU LLVM performance improvements (#5882)

Next up are some much needed performance improvements to the SPU LLVM recompiler by Nekotekina. A trick was used to optimize LLVM’s LICM (loop-invariant code motion) pass, giving up to 10% improvement in FPS in many games when using accurate xfloat. Approximate xfloat was however unaffected by this change.

To understand how Nekotekina achieved this, we first need to understand what the LICM pass does. Simply put, it checks all the loops in the code for parts that don’t change (loop-invariant) and moves them out of the loop (motion) so they are not unnecessarily executed over and over again. This can be a major optimization factor in some cases, however LLVM doesn’t always apply this pass perfectly. That is why Nekotekina decided to improve this by replacing some instructions that caused LLVM to not correctly identify these loop-invariant sections with some dummy instructions that are known to not interfere with the LICM pass. So after LLVM did its job we can once again replace those dummy instructions with the real ones, leaving the code functionally equivalent, yet better optimized.

Captain America: Super Soldier saw an increase in FPS from 20 to 24 in slow areas!

New Skyline and Envy themes (#5789, #5884)

For those of you that didn’t know, RPCS3 has given users the ability to customize the appearance of the emulator for quite a while now and since then, a few custom themes have been added to the roster.

This month, two new stunning themes have been added by drysalter in his first ever contribution to RPCS3. The Envy theme sports a completely new neon design giving users another dark theme to use while the Skyline themes are inspired by DAGINATSUKO’s colour scheme of the website and comes in two variations: Standard and Nightfall. You can find these and other cool themes by navigating to Config > GUI > UI > Stylesheets. Just choose one, click Apply and instantly see RPCS3 in a new light!

Envy	Skyline	Skyline Nightfall

DualShock 3 support on Linux (#5888)

Lastly, let’s take a quick look at GalCiv’s (RipleyTom) improvements to the native DualShock 3 pad handler. As we covered in last month’s report, GalCiv implemented a native DualShock 3 pad handler that allows emulation of features such as pressure-sensitive buttons and motion controls in RPCS3 when using the DualShock 3 controller. While this change was much welcomed, the implementation was sadly restricted to only Windows as the use of generic USB drivers required special consideration on Linux.

GalCiv, not wanting to leave the job half done, looked into implementing the same on Linux when he made an interesting revelation. The Linux kernel has the drivers for the DualShock 3 controller built-in. This allows direct support for the controller without the need for solutions used on Windows.
Taking advantage of the same, GalCiv implemented a DualShock 3 handler on Linux utilising this driver, providing the full range of features such as pressure-sensitive buttons and motion controls through USB or Bluetooth without the use of any third-party devices. Fascinating to see what started as a deficiency turn into yet another resounding win for team Pingu!

Games

Sonic Unleashed

Thanks to Eladash’s improvements to the PPU LLVM recompiler, this game no longer hangs during night stages and when doing quicksteps. Performance has also vastly improved during night stages, but it’s still too slow to be considered Playable. Check out the video footage from our official YouTube channel below:

NCAA games

Let’s begin with some sport games. NCAA Football 09 and NCAA Football 10, two football games are now playable this month. Both games are running well with good graphics and performance.

Madden games

Madden NFL 11 is the first game in the Madden series to have been moved to the Playable category after Nekotekina fixed some regressions this month, while Madden NFL 13 and 09 are now in the Ingame category.

NBA games

Most NBA games, for example NBA 2K7 and NBA 2K8, are now going ingame for the first time. However, both titles are too slow to be considered playable at this point.

MLB games

Moving on to baseball games, MLB 08: The Show and MLB Front Office Manager both became playable this month.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

5784 – Eases cache line interference on TSX path, emulates POSIX unlink on Windows and extends spurious access error workaround to all directory renames;

5822 – Disables GCC build for Travis, improving overall build time;

5823 – Fixes some regressions caused after #4097 and #5784;

5832 – Updates LLVM submodule and increases max stack size to avoid stack overflow crash on Windows;

5844 – Minor refactoring to the SPU;

5855 – Makes a refactoring to the LLVM DSL;

5882 – Improves SPU LLVM performance when Accurate xfloat is enabled. See coverage in major improvements here.

kd-11

5785 – Further improvements to the texture cache. See coverage in major improvements here;

5813 – Fixups to some regressions caused by the previous PRs in last month. See coverage in major improvements here;

5860 – Improvements made for the shader decompiler, see coverage in major improvements here;

Megamouse

5809 – Adds a new hint to the game list that shows the highest available version if the installed version of that title is lower than the one that was found in the compatibility database.

5780 – Adds more filters to firmware installation file dialogs and enables the search and installation of firmware files that are named differently than PS3UPDAT.PUP.

5666 – Prints the currently installed firmware version into the first line of the log.

5763 – Adds a missing entry for the recently added 3rd party software libusb into the Visual Studio solution files.

5687 – Adds an “Exit RPCS3?” dialog if the emulator is being closed while a game is running. This dialog uses the same option in the gui config tab as the “Exit Game?” dialog. The confirmation dialog will always be on top to prevent it from disappearing behind other windows. From now on the game window will also properly leave fullscreen mode if an “Exit game?” pop up is triggered when the game window is being closed.

elad335

5749 – Several improvements for the PPU and SPU interpreters and recompilers, see coverage in major improvements here;

5861 – Sets the performance overlay detail to Medium by default because the RSX Guest utilization confuses people and it’s only meaningful for debugging.

5792 – Various improvements and bugfixes for cellSaveData module, see coverage in major improvements here;

5871 – Adds missing SDK version check for setParam->reserved2 in cellSaveData module.

5850 – Fixes unregistered HLE function access on PPU LLVM and PPU breakpoints, see coverage in major improvements here;

scribam

5790 – Adds some missing functions for cellCelpEnc, cellHttp and sceNpMatchingInt modules;

5797 – Adds more missing functions for several modules;

5814 – Adds some other functions for several modules.

MSuih

5819 – Fixes compiling for MSVC and CMake;

5851 – Drops severity of empty queue error on sys_net_bnet_close from fatal to error, and fixes the stack size argument. Previously, it was spamming unknown option on every compiled file. Only affects people using MSVC with CMake;

5870 – Prints the user’s operating system version to the log.

drysalter

5789 – Add Envy and Skyline themes, see coverage in major improvements here;

5884 – Touch-ups to Skyline and Envy themes, see coverage in major improvements here.

Inviuz

4007 – Provides initial sys_overlay support which is used in Metal Gear Solid 4, see coverage in major improvements here.

GalCiv (RipleyTom)

5888 – Reimplements part of DS3 pad handler to make it work with hidapi for Linux, see coverage in major improvements here.

AniLeo

5873 – Changes vendor and product IDs from the Rock Band series to use the Guitar Hero ones and adds vendor and product IDs from the DJ Hero Turntable and the Dance Dance Revolution Mat.

TGEnigma

4097 – Adds extra argument checks for _sys_ppu_thread_create and sys_ppu_thread_rename and pause the thread instead of throwing an exception when reaches a trap instruction (useful for debugging).

Johndeep

5897 – Updates the Qt dependency for Debian and Ubuntu in the README file.

vieiraa

5840 – The languages available in the combo box in the system settings will appear sorted alphabetically instead of being sorted by internal ID.

JohnHolmesII

5794 – Fixes a Travis deployment bug for Linux builds.

al3xtjames

5782 – Fixes broken macOS compilation.

violettte

5775 – Updated the domain name of the Vulkan mirror for our Appveyor build.

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. RPCS3 now has two full-time coders that greatly benefit from the continued support of over 800 generous patrons.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by MarioSonic2987, elEnemigo and HerrHulaHoop.

In addition to the following report, further details of Nekotekina and kd-11’s work during March and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-03-02)
Status update from Nekotekina (2019-03-05)
Status update from kd-11 (2019-03-18)
Status update from Nekotekina (2019-03-30)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

This month saw more maintenance work done to the compatibility list where duplicate game IDs for the same game were merged into one single entry. A grand total of 118 threads were merged allowing for a fairer representation of RPCS3’s compatibility at this time. Adding to this, our community of testers continued their efforts in acquiring and testing obscure titles that had not been tested recently. The results of this revealed that many of the titles in the Ingame and Intro categories were now indeed Playable and moved according. Thanks to the dual effect of both efforts, the Playable category saw a marginal increase while the Ingame and Intro categories saw a sharp decrease in total number of games listed. Finally, thanks to elad335’s improvements to sys_vm, Doom 3: BFG Edition moved from Nothing category straight into the Playable category, leaving just two more titles in the Nothing category! For a more detailed look, you can view the compatibility history page to see exactly which games had their status changed this month.

On Git statistics, there have been 6331 lines of code added and 2573 removed through 105 commits by 9 authors.

Major RPCS3 Improvements

RSX Texturing improvements (#5675, #5726, #5741, #5752)

This month, kd-11 was focused on work that aimed to properly preserve rasterized data without falling back to slow synchronization techniques. From the roadmap shared with patreons, the first step towards that is to build the foundation for smooth implementation of features by rewriting the texture cache to improve how texture data is interpreted for sampling. This was an arduous task as past experience showed us that many games are extremely sensitive to texture cache changes and often regress when modified. However, with the help of our dedicated tester base testing every game at their disposal, kd-11 was able to rewrite the texture cache and seamlessly address most regressions. For those looking for a detailed look into what the texture cache is and how it works in RPCS3, you can read our “Quick” Primer on Texture Caching in RPCS3.

Over six weeks, kd-11 hammered the texture cache into shape by going for an accuracy focused approach with regards to advanced synchronization and data resolve across the separate surface management cache and the texture data cache. The primary objective was to improve the texture search logic by attempting to ‘merge’ the surface store data and the texture cache data whenever overlaps are encountered. Due to performance concerns, a full data collapse would be infeasible but since the two sources are optionally data locked, this should almost always resolve correctly especially when Write Color Buffers is enabled.

The Last of Us saw a significant improvement to indirect lighting

After this, he immediately began working on getting the blit engine to behave correctly with the new structures and policies implemented over the rest of the cache. The blit engine is the 2D rendering engine inside the GPU, capable of handling memory-intensive tasks such as asynchronous memory copies over DMA, texture scaling, format conversion, fast texture data copies as well as rotations around the X or the Y axis which can be thought of as ‘flipping’ the image. This work builds upon the previously existing work as the initial implementation broke many games when using the hardware (GPU) to accelerate these operations, necessitating the use of the ‘CPU blit’ debug option to get the correct output. Apart from this, few more features such as pitch compatibility methods were also implemented but have not been enabled yet. These features will be enabled down the road once the rest of the code is stable enough to accommodate it.

All of this paid off with graphical improvements across a significant number of games and fixed long-standing issues such as flickering in Demon’s Souls, water reflections in Ratchet & Clank: A Crack in Time as well as the Atlantica stage and Genie Jafar stage in Kingdom Hearts HD 1.5 ReMIX (removing the need for Strict Rendering Mode and allowing for resolution upscaling). Games such as Beyond: Two Souls and inFamous: Festival of Blood no longer need CPU blit to properly render textures leading to a significant performance boost as well. Finally, with these changes, Saints Row: The Third and Saints Row IV render graphics for the first time in RPCS3.

New LLVM-based SPU Interpreter & SPU improvements (#5762, #5721, #5770)

This month, Nekotekina overhauled the SPU Interpreter (fast) into an LLVM-based Interpreter that shares the same backend as the current LLVM Recompiler. This was an important step as the unified the backend for the interpreter and recompiler eliminates any accuracy differences between the two implementations and also simplifies the underlying code. Added to this, the interpreter now respects the xfloat accuracy options that were previously restricted only to the LLVM Recompiler and can also benefit from architecture-specific code generation by LLVM.

In terms of performance, the new interpreter saw massive performance improvements over the legacy implementation with some games titles achieving a 20% increase in FPS. Furthermore, in some titles such Sengoku Musou 4 and Dynasty Warriors: Gundam 3, the FPS when using the interpreter improved performance from 10FPS to 50FPS, surpassing even the SPU ASMJIT Recompiler!

This was the first step in Nekotekina’s SPU/PPU roadmap given to patreons and he is hard at work improving the synergies between the two backends and using it to yield much greater performance and accuracy. If you wish read the full PPU/SPU roadmap by Nekotekina, you can check his patreon post.

Besides this, Nekotekina also improved the MMIO method handling in the emulator. During debugging, elad335 identified that SPU LLVM emits some MFC code in CPUs with TSX enabled and some checks for address in SPU MFC are MMIO but treated as usual DMA leading to issues. In the context of SPU MFC, DMA (Direct Memory Access) is a method used to read/write data from/to the SPU local cache into the global memory. Whereas, MMIO (Memory-mapped I/O) is used to read/write to a register and also perform an action in addition to data access. To prevent SPU LLVM from incorrectly using DMA for these checks, Nekotekina implemented a fallback to the general MFC MMIO handler that will correctly handle such transactions. Not only does this improve the accuracy of emulation, but in cases like raw SPU MMIO, the check was previously handled by the PPU access violation handler which is known to be expensive. Now, these checks also use the general MFC MMIO handler leading to a notable speed increase.

Finally, Nekotekina also bumped the version of LLVM used in RPCS3 from 7.0 to 9.0 and implemented use of few new features offered by the latest version. While LLVM 9.0 is still in the research phase and has not been fully released, Nekotekina opted to adopt it early and utilise the benefits that it provides.

Support for DualShock 3 controllers (#5641)

As with every PlayStation console, the PlayStation 3 was released along with a new DualShock controller, the DualShock 3. This controller was a great leap forward as not only did it still have pressure-sensitive buttons from its predecessor, it also introduced motion control technology (SIXAXIS as Sony called it). This was a large step ahead from other controllers that lacked pressure-sensitive buttons and SIXAXIS but the exclusivity of these features also led to rather low adoption-rate from game developers. That’s not to say they weren’t used, many first party or PlayStation 3 exclusives made use of these features such as Metal Gear Solid 4, Flower, Heavy Rain, Uncharted 1 and Heavenly Sword.

Since the DualShock 3 was made for the PlayStation 3, its drivers are incompatible on PC and need third-party tools to work. While most third-party tools used to extend support for the DualShock 3 worked well with RPCS3, their implementations caused some notable issues. Most commonly used tools behave as wrappers around the standard XInput API on Windows, converting input from the DualShock 3 to that which is compatible with XInput. However, since XInput had no functionality for motion controls or pressure-sensitive buttons, such inputs from the controller are simply ignored. This called for a dedicated pad handler for DualShock 3 to effectively support its full range of features.

Now, anyone who has followed RPCS3 knows that it has a dedicated pad handler for the DualShock 4 since May 2017. The DualShock 4 shared a lot of similarities with its predecessor such as being designed to work solely with the PlayStation 4, having a range of unique features not present in other controllers and requiring third-party software to work on PC. So why was the implementation of the native pad handler much simpler compared to the DualShock 3? To put it simply, the DualShock 3 suffers from certain fundamental differences that prevent it from being supported as easily.

Most controllers interact with host systems through the USB port and adhere to the USB HID (Human Interface Device) class defined by the USB Implementers Forum. As per this specification, every device describes how it will communicate with the host system to allow the host system to accurately predict and define all current and future interactions with the device. During enumeration the device describes how its reports are to be structured so that the host system can properly prepare to receive this information. The creation of this standard enables Operating Systems to implement a standardised framework for HIDs and allows for seamless integration with various devices. Similarly, the Bluetooth HID class is a simple wrapper around the USB HID class simplifying OS implementation. So if you ever wondered who thank when your printer, mouse and keyboard just work when connected for the first time, now you know!

All controllers (including the DualShock 4) adhere to this USB HID class and communicate with consoles/PCs using this specification. However, the DualShock 3 deviates from the specification and requires the host system to send a GET_REPORT request to start communication. Since the report is not advertised by the DualShock 3, the OS does not allow us to send this request to it. To tackle this deficiency, GalCiv implemented a system where the controller is first recognised by the OS as a generic USB device and then subsequently handled by RPCS3 as a controller to allow input to games. While this approach has allowed us to support DualShock 3 and all of its unique features, it does come with a few requirements. Firstly, users are required to install generic USB drivers (like WinUSB) to access the controller. Since the controller is required to be recognised as a USB device, Bluetooth connectivity is not supported at this moment. Finally, for the time being, native support for DualShock 3 is restricted only to Windows. With this, RPCS3 finally natively supports the DualShock 3 allowing for better emulation of titles that utilised the controllers unique offerings!

sys_fs (fsync) Improvements (#5506)

Continuing his bug fixing spree, elad335 made a wide variety of improvements to the emulator this month. Only 5 pull requests were made but each of these contained multiple fixes for a wide range of components. While most of these fixes are significant, not all translate directly to games; some improvements can be seen in multiple titles while some fix specific issues to a single game but others don’t show any immediate effects in games. If we were to list all these changes and explain why they were made, this report would never see the light of day. So here’s a quick look at one fascinating change made by elad335 this month. For the full list of improvements, see the Other Improvements section below.

While debugging a few AAA titles, a startling discovery was made. Two users tested God of War 3 with similar hardware but one user barely managed to hit 20FPS while the other was comfortably at 40FPS. Investigating why such a drastic discrepancy existed on similar hardware, a diligent check uncovered that one user had the game on an SSD while the other had it on a HDD. Now this may seem reasonable at first glance, it is anything but. The PlayStation 3’s Blu-ray drive operated at a maximum speed of 9 MB/s while the internal hard drive was able to transfer data at upto 100 MB/s (depending on the model). Both these numbers are much lower than PC HDDs and leagues below SSDs. This is why the users barely find any difference in performance when running game on RPCS3 from either a HDD or an SSD. Even the cache generated by RPCS3 is loaded into the RAM/VRAM while booting games nullifying any effect in this regard. So why did God of War 3 require an SSD to operate at maximum performance?

A big clue to answering this question came when one of the testers decided to run the game on Linux (since all previous metrics were obtained only on Windows). Running God of War 3 on Linux, we were easily able to reach the 40FPS mark at the test spot, only this time the game was still running on a HDD. This result immediately pointed to some special handling in Windows for HDDs as opposed to a bug in RPCS3. God of War 3 was clearly triggering something that was handled easily on Linux but choked Windows with the same HDD. Investigating further, elad335 identified that the issue was due to frequent file cache buffers flushing by the game.

What does file cache buffers flushing do you ask? Generally, an operating system stores data in file read and write operations in system-maintained data buffers to optimise disk performance. When an application writes to a file, the system usually buffers the data and writes the data to the disk on a regular basis. However, an application can force the operating system through an fsync transfer to flush contents of these data buffers to the disk. Some HDDs also have dedicated in-built cache for further buffering and Windows goes an extra step and requires the disk to clear this cache as well. Buffers are flushed mainly to ensure a consistent state of written files in the case of power loss events and similar disasters.

Coming back to the problem at hand, for some reason (best known only to SCE), God of War 3 requests for a file buffer flush after every 1MB written to disk. This was handled fairly well on Linux but proved expensive on Windows and at such a high volumes, crippled performance as PPU threads would wait for this operation to complete before yielding. elad335 addressed this by yielding the PPU thread before performing the buffer flush so the emulator was free to reschedule the thread immediately for the next task. This solution brought parity in performance between both operating systems.

A comparison of performance in God of War 3, running on Windows

VSH Progress

We’re wrapping up this month’s coverage with a quick update on running VSH in RPCS3. Back in November of last year, we showcased work done by ruipin and many developers to reach XMB menu (PlayStation 3’s GUI) in RPCS3. At the time, users were required to dump xRegistry.sys from their PlayStation 3 to allow VSH to boot. As it turns out, ruipin has been chipping away at VSH to see what makes it tick. After extensive reverse engineering, he finally managed to implement the sys_config syscalls, one of the key features required for RPCS3 to be able to boot VSH standalone. VSH uses sys_config as a way to permanently store system settings and hardware configuration, e.g. how many controllers are connected. Without these syscalls implemented, VSH is unable to identify connected controllers, and as such does not recognize any input.

With this advancement made in the VSH branch, RPCS3 can now boot VSH and proceed past the initial system configuration and to XMB without the xRegistry.sys file. While the VSH branch is not yet ready to be merged into the master branch, the work done on sys_config is mostly complete and ruipin has opened a pull request to have the same merged into master. For those of you interested in the technical description of sys_config and its implementation, click here.

If you’re interested to check out the XMB menu running on RPCS3, refer to the instructions here. Please note that while the developers have made large strides to boot VSH, it still has a long way to go before it is ready for endusers. In its current state, it cannot be used to boot games installed in RPCS3. However, PlayStation 3 themes that are installed in RPCS3, can be applied and used in the XMB menu!

Games

Gran Turismo 6 & HD Concept

This month saw a large number of racing titles improve in performance and stability and with the PlayStation 3, you can’t mention racing titles without the Gran Turismo series. Since first going ingame in November 2018, the Gran Turismo series has been steadily improving in the graphics and stability departments. Thanks to the efforts from various developers in the past few months and elad335’s implementation of SPU page faults this month, Gran Turismo HD Concept has finally become playable! Check out the extended gameplay footage below:

And if that wasn’t enough, thanks to the page faults notification fixes by elad335, Gran Turismo 6 (and the Ayrton Senna Special Edition) now progresses ingame as well! However, being the latest instalment in the series on the PlayStation 3, it does currently suffer from heavy graphical issues. The game also suffers from low performance due to requiring PPU Interpreter to progress ingame. However, work has already been undertaken to improve compatibility of PPU LLVM which will significantly boost performance in such titles.

Supercar & Ferrari series

Multiple racing titles from Eutechnyx studio went in-game for the first time in January. Thanks to the stability improvements made to the emulator in the past two months, Absolute Supercars, Supercar Challenge, Ferrari Challenge: Trofeo Pirelli and Ferrari: The Race Experience are now comfortably playable!

Ridge Racer 7

Continuing the racing title improvements, the PlayStation 3 exclusive Ridge Racer 7 is now Playable! Previously, this title suffered from low performance and a few graphical issues that have now been fixed. Hopefully the first of many Ridge Racer titles to become playable on RPCS3.

NASCAR Unleashed

To finish off the racing coverage in this month’s report, NASCAR Unleashed also made its way to playable. This console exclusive had escaped testing for over a year and previous only displayed a black-screen when ingame. These issues have now been summarily fixed.

Doom 3: BFG Edition

As most of you are already aware, Doom 3: BFG Edition was one of the three notorious titles still stuck in the Nothing category in RPCS3’s compatibility list. Thanks to improvements made to sys_vm by elad335, this title went from Nothing straight to Playable!

College Hoops 2K7

Moving on to sports titles, College Hoops 2K7 went ingame for the first time this month. While the title has good graphics, it suffers from low performance causing issues with the game physics keeping it from being playable on the emulator.

Move Street Cricket I & II

Finally, this month saw both instalments of Move Street Cricket make their way to playable. If you’re a fan of cricket, these games are definitely worth trying on RPCS3.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

5704 – Fixed bug relating to sys_fs_opendir on root directory; Updated sys_lwmutex_lock and sys_lwmutex_unlock (liblv2 HLE) to implement missing SYS_SYNC_RETRY logic; Improve LLVM recognition of Intel Cascade Lake CPUs;

5715 – Updated third-party submodules such as pugixml, xxHash, yaml-cpp and libpng along with other minor optimisations;

5721 – Fixed handling of SPU MFC code in certain scenarios when using the SPU LLVM recompiler, see coverage in major improvements here;

5735 – Fixed a regression caused by PR 5599 where certain games would crash with an Out of memory error when using the SPU ASMJIT recompiler;

5762 – Introduced new LLVM-based SPU interpreter (fast), see coverage in major improvements here;

5770 – Updated LLVM submodule from version 7.0 to version 9.0, see coverage in major improvements here;

kd-11

5675 – Improved RSX texture search logic, see coverage in major improvements here;

5726 – Improved implementation of texture cache and blit engine, see coverage in major improvements here;

5741 – Improved implementation of texture cache, see coverage in major improvements here;

5752 – Improved implementation of texture cache, see coverage in major improvements here.

elad335

5677 – Fixed regression affecting RSX capture replay and improved accuracy of sys_rsx error checking;

5697 – Fixed PPU scheduling in cellSaveData when an external GUI function calls to show saves list allowing games to continue background music and graphics while showing the saves overlay. Also fixed a few deadlocks after loading or storing saves in games such as Grand Theft Auto: San Andreas and Transformers; Fixed a race condition in Vulkan ZCull management regarding access violation handler. This addressed instability issues seen in titles from the Yakuza series when ZCull is enabled; Fixed a race condition in sys_event_flag_set regarding setting flags; Fixed unknown/unimplemented X8R8G8B8 format in image_in using A8R8G8B8 handler; Fixed a regression in Gran Turismo 5 from the above pull request by further improving sys_rsx error checking accuracy by fixing page flags on mirrored memory;

5506 – Yield before flushing io buffers in fsync (sys_fs), see coverage in major improvements here;

5668 – Implemented SPU page fault notifications and fixed various aspects related to page faults such Implementing both RawSPU and threaded SPU page fault recovery, guarding page_fault_notification_entries access with a mutex, adding missing lock in sys_ppu_thread_recover_page_fault/ get_page_fault_context, fixing EINVAL check in sys_ppu_thread_recover_page_fault, previously when the event was not found begin() was erased returning CELL_OK and many more. This fixed the save issue in Gran Turismo 5 and allows Gran Turismo 6 to go ingame for the first time on RPCS3!

5719 – Fixed memory statistics changes of sys_vm, allowing memory statistics to increase or decrease according to action, implemented ENOMEM (not enough memory) and other error checks for sys_vm syscalls. This change fixed Doom 3: BFG Edition allowing it to become playable! Fixed a leak in cellVdec handle management that caused games to crash after 256 cutscenes or other video playback using cellVdec; Fixed segfault on viewing stack memory addresses on the debugger; Fixed picture information packet status check of cellVdecGetPicItem. This fixes Toy Home and SIREN: Blood Curse, allowing both titles to now be fully playable! Fixed a corner case in sys_process_exit2 (exitspawn) where SPUs use the access violation handler and are unable to quit from it. Fixed deadlock in the game selector of Yakuza 1 & 2 HD; Fixed timer state after event queue attached to the timer has been destroyed externally; Fixed page faults notification range check; Fixed flags of device set in cellPadGetInfo; Fixed a race condition in sys_timer_destroy that can result in a deadlock where the timer thread times-out at the same time; Optimized RSX FIFO registers control and cellPad functions; Fixed PPU scheduling in sys_timer when no sleep time has been specified.

GalCiv (RipleyTom)

5696 – As mentioned in last month’s progress report, some of the latest Guitar Hero games check the VendorID/ProductID to determine whether the connected peripheral is a Guitar Hero guitar controller or drum kit. This commit specifies the same through cellPadGetInfo;

5718 – Fixed an issue with lf_queue destructor which caused crashes under certain conditions;

5641 – Implemented a native DualShock 3 pad handler in RPCS3. See coverage in major improvements here;

scribam

5756 – Added function names for HLE cellNetAoi and sceNpMatchingInt modules;

5759 – Added function names for HLE cellDtcpIpUtility, cellPesmUtility and cellVideoPlayerUtility modules;

5761 – Added function names for HLE cellSysutilAvcExt module;

5765 – Added function names for sceNpEulaAbort function;

5767 – Added function names for HLE libad_async and libad_core modules;

5771 – Added function names for HLE cellFontInitLibraryFreeType, cellSysmoduleUnloadModuleInternal and cellSysmoduleLoadModuleInternal modules;

AniLeo

5716 – Fixed a fatal error that occured when installing custom firmware without \n in the version string.

RainbowCookie32

5734 – Updated the tooltip in the Pad configuration dialog to reflect native support for DualShock 3 controllers.

jbeich

5738 – Unbreak build on FreeBSD.

Jocker666z

5774 – Added qtdeclarative5-dev to fix compilation on Debian and Ubuntu.

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. RPCS3 now has two full-time coders that greatly benefit from the continued support of over 800 generous patrons.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, MarioSonic2987, elad335, GalCiv and Digitaldude555.

In addition to the following report, further details of Nekotekina and kd-11’s work during February and upcoming contributions can be found in their reports on Patreon. This month’s Patreon reports are:

Status update from kd-11 (2019-02-08)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

This month saw RPCS3 reach another milestone in game compatibility with the Ingame and Playable categories cumulatively crossing 85%! This was largely thanks to the conscious efforts of testers to retest all games in Intro, Loadable and Nothing categories and report issues faced to the developers. Once provided relevant causes, our developers were able to fix issues plaguing many of these titles. Their efforts paid off with the Intro category decreasing by 66 titles. But this is just the beginning, with further work being done to improve these titles.

On Git statistics, there have been 2209 lines of code added and 447 removed through 21 pull requests by 9 authors.

Major RPCS3 Improvements

Support for PlayStation 3 accessories (#5691)

Like any console, the PlayStation 3 had a host of accessories and toys that could be connected for additional functionality. This include accessories such as the Guitar Hero guitar controllers, Rock Band drum kits, SingStar microphones, Logitech Driving Force GT racing wheel and even toys such as the Skylander Portal. Sony also released their own set of peripherals such as the PSEye camera and PSMove motion controller along with the DualShock 3. While all peripherals connect to the console through the USB port, they are handled differently by each game. The PlayStation 3 allows developers to handle these peripherals by two methods, either through specialized libraries (such as cellPad, cellMic, cellGem and cellCamera) or by communicating directly with the USB device through cellUsbd.

In RPCS3, the cellPad library was the first to be implemented along with the Pad handler (using input APIs such as XInput, MMJoystick, evdev) and have games detect input as if it were from a DualShock 3. Using these APIs ensured that RPCS3 would be able to recognise any controller that complied with these APIs. While the other libraries have also been implemented to varying degrees, more work is required for RPCS3 to support these peripherals.

Or so we thought, until AniLeo, our community manager, noticed that he could get Guitar Hero 5 to recognise his GH guitar controller using the MMJoystick API by forcing cellPad to return a value (through cellPadPeriphGetInfo & cellPadPeriphGetData) indicating that the connected pad was a guitar. The erstwhile implementation of cellPad passed all inputs received as if it were inputs from a DualShock 3. AniLeo then passed this information on to GalCiv who quickly implemented a simple drop-down menu in the Pad configuration to allow users to select the type of peripheral they wish to emulate.

While this allowed few games to seamlessly detect the guitar controller, other titles still failed to detect the controller correctly. GalCiv promptly began investigating the issue and after reverse-engineering these titles, he identified that they were checking for specific VendorID or ProductID of the controller to allow inputs. Forcing these values to that of Guitar Hero guitar controllers and drum kits addressed these issues as these peripherals are supported by both Guitar Hero and Rock Band titles. With this, users can now play Guitar Hero games with the gamepad of their choice (emulated as a pseudo guitar controller) or use the MMJoystick API to connect an actual guitar controller and enjoy an authentic experience!

With these improvements, many titles such as Rock Band 2 correctly recognise input and are now fully playable!

Users should be note that these improvements only benefit titles that detect input using the cellPad library. A few titles (most notably the latest Rock Band games) use cellUsbd, instead of cellPad, to directly communicate with the guitar controllers and drum kits. However, not one to leave a job unfinished, GalCiv is now hard at work on implementing microphone support through cellMic and also sys_usbd (lv2 syscalls used by cellUsbd). Once finalised, users will be able to use peripherals such as the Skylander Portal and SingStar microphones while also being able to emulate regular microphones as pseudo SingStar microphones!

Native OSK (#5611, #5631, #5633)

Back in January 2018, kd-11 decided to implement a native UI for RPCS3 which allowed overlay interface elements to be displayed on top of games. Prior to the native UI, these interfaces were implemented through Qt dialog boxes that would pop-up during gameplay. While these pop-ups gave users the necessary functionality, it suffered from certain drawbacks. For one, these dialogs can only be interacted with using keyboard and mouse, so users looking for a console experience were left with much to desire. The second issue was the fact that, when in fullscreen mode, the message dialogs would in fact appear behind the game and users were given the impression that the game was stuck when it was simply waiting for input. To address these issues, kd-11 implemented a native user interface within the same context frame allowing users to interact with various interfaces such as the save/load menu and message dialogs using a controller.

While most of the functionality was ported over to the new native UI, the On-Screen Keyboard interface was excluded as it required significant rework to the existing implementation. On-Screen Keyboard functionality was implemented with the Qt dialog interface in January 2017 but required users to use a physical Keyboard to input text when prompted. Since the native UI did not implement the same, users were unfortunately still denied the console experience. However, not wanting to leave the task half done, Megamouse improved the implementation of both the Qt dialog interface as well as the native UI interface culminating in the large-scale refactoring last month, all with the goal of facilitating a smooth implementation of a native On-Screen Keyboard in RPCS3.

Once this exercise was completed, kd-11 fixed the few remaining bugs and implemented an On-Screen Keyboard interface within the native UI. The first step involved designing and testing the new On-Screen Keyboard to ensure feature parity to the existing Qt implementation. The initial implementation was made supporting ironing out the design while supporting English.

However, the native UI had a long standing drawback of only supporting characters from English. With this limitation extending to On-Screen Keyboard, latin characters (such as áëéñüùç) and Japanese glyphs were not rendered and instead simply replaced with blank spaces. To workaround this limitation, kd-11 stripped all unsupported character and replaced them with the ‘#’ sign. Next, he upgraded the text rendering system to use extended ASCII character encoding allowing many more characters to correctly render. This allowed for text in languages such as French and Spanish to correctly render now.

Characters from unsupported languages like Japanese will be demarcated with the ‘#’ sign and users will be required to use the erstwhile Qt implementation for full compatibility. However, further improvements were made to the native UI to reduce the number of ‘#’ sign being displayed by identifying spaces and punctuations in full-width languages such Chinese, Japanese and Korean and mapping them correctly to their half glyph counterparts.

Among these changes, the quality of text rendering (especially with OpenGL) was also significantly improved

Games

AC/DC Live: Rock Band, Rock Band and Rock Band 2

Thanks to GalCiv (RipleyTom) implementing support for guitar and drum peripherals Rock Band, Rock Band 2 and the Track Pack games such as AC/DC Live: Rock Band Track Pack are now playable!

MotoGP 10/11

Continuing the recent surge of improvements to racing titles, MotoGP 10/11 is now fully playable!

WWE SmackDown vs. RAW 2009 and WWE ’12

Among the few WWE titles to miss out on a PC port, WWE SmackDown vs. RAW 2009 and WWE ’12 now go ingame. However, both titles suffer from stability issues keeping them from being playable. If you wish to try WWE SmackDown vs. RAW 2009, do remember to turn off wrestlers’ entrances in the game menu.

Tokyo Jungle

The PlayStation exclusive title, Tokyo Jungle, is now fully playable! Previously, this title would freeze shortly after going ingame but with recent compatibility improvements, this title now with runs at full speed even on mid-range hardware. Check out some gameplay footage below:

GripShift

This month also saw the action racing title GripShift move to playable. This console exclusive previously suffered from broken graphics and poor performance but recent improvements have fixed all such issues. Do remember to set framelimiter to 60FPS, as this title now runs well over 200 FPS!

Genji: Days of the Blade

When Genji: Days of the Blade went in ingame in late 2017, it suffered from various graphical issues and the characters were stuck in a T-pose. However, thanks to all the improvements made to the emulator since then, this title performs much better. While performance has improved, further testing is required to determine whether this title is indeed playable. In the meantime, check out this title’s current performance here:

MX vs. ATV Alive

MX vs. ATV Alive went ingame for the first time this month. This marks the milestone of all titles from the off-road racing series MX vs. ATV to now go ingame. Sadly, the game suffers from severe graphical glitches keeping it well away from the Playable category.

Dark Mist

Yet another PlayStation 3 exclusive, Dark Mist, now progresses Ingame. Do remember to enable LLE libvdec, Write Color Buffers and Strict Rendering Mode to avoid significant issues.

Super Hero Generation

A spin-off involving character from three notable franchises, Gundam, Kamen Rider and Ultraman, this title was found to go ingame this month. Further testing is necessary to ascertain whether it is playable on RPCS3.

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details as well as the code changes themselves. To see this whole list right on GitHub, click here.

Nekotekina

5693 – Added framework for future replacements to IDM and FXM; Optimized assembly utility instruction for few compilers; Fixed a race condition in sys_spu_thread_group_join during concurrent access; Improved SPU branch patchpoints to now work with SPU LLVM in addition to SPU ASMJIT.

kd-11

5611 – Implemented native On-Screen Keyboard (OSK). See coverage in major improvements here;

5631 – Added support for extended glyph sets in the native system overlay. See coverage in major improvements here;

5633 – Further improvements to render quality of system overlays. See coverage in major improvements here;

5635 – Fixed usage of VK_IMAGE_LAYOUT_GENERAL by properly synchronizing transitioning to and from General layout. General layout requires full pipeline dependency due to its general usage and as such, should be largely avoided.

GalCiv (RipleyTom)

5285 – Implemented static function hooking for memcpy, memset, memmove and memcmp. This can be activated by enabling Hook Static Functions in the Debug tab of the Settings menu;

5691 – Added support for various PlayStation 3 accessories such as Guitar, Drum, Dance Mat, etc. See coverage in major improvements here.

elad335

5623 – Fixed gaps in error checking of lwcond signal syscalls. Also fixed mode 2 of signal syscalls which did not work correctly. Now the error checking correctly matches the lwcond syscalls’ library wrappers;

5643 – Unlike usual threading norms, SPU groups initialization process can be terminated infinitely instead of once. Before this improvement, call join could be done infinitely after a termination without waiting.
This was discovered to be incorrect as once a join was called, the termination signal is cleared and another signal need to sent before the next join could finish as well. This has now been implemented accurately which also simplified the termination process;

5645 – Fixed a minor race condition introduced by the above pull request. When the SPU group is in an “almost” finished state before sys_spu_thread_group_join is called, there was a small time gap where calling the syscall would return from execution before all threads are terminated completely;

5667 – Fixed a race condition when error codes are returned and the error value is dependant on external access in a specific mode of the syscall. This mode did not work prior to PR 5623 and hence the race condition was concealed. Once the mode was correctly implemented, the race condition manifested as a perceived regression in Demon’s Souls which was fixed as well;

5680 – Implemented SYS_SYNC_RETRY lwmutex protocol logic which improved LLE liblv2 compatibility. This benefitted titles such as Super Robot Taisen OG Infinite Battle.

Megamouse

5542 – Improved accuracy of cellSysCacheMount considering the scenario when cacheId is empty and system cache is supposed to be cleared. This improved allowed NBA 08 Demo to progress from Loadable to Intro!

5581 – Improved cellGame error dialogs by adding the relevant error_code and improving the error messages;

5620 – Fixed a bug where the “Open custom config location” option pointed to the previously used folder instead of the current folder;

5650 – Improved crash dump functions in sysPrxForUser which addresses crashes that appear during TODO logging;

5651 – Only show supported resolutions options in the Settings dialog of per-game configurations

5678 – Removed the Viewport options from Settings dialog of per-game configurations as the Viewport settings are only taken from global configurations.

gasinvein

5656 – Improved appdata.xml by adding URLs to the GitHub issue tracker and Quickstart guide, and a screenshot of the emulator. RPCS3 now passes appstream-util validation.

al3xtjames

5657 – Improved macOS build compilation by using the current Git tag to generate version strings instead of using hardcoded values.

jbeich

5695 – Fixed an error in the above pull request.

EmulationChannel

5665 – Updated the latest firmware version to 4.84.

GermanAizek

5670 – Fixed use of clear() to create empty strings.

JohnHolmesII

5682 – Fixed link to Qt installed in the README.md.

Upcoming

Apart from the amazing work by GalCiv to support more peripherals, elad335 has also been working on improving compatibility of the PPU LLVM recompiler. With games being able to fully utilise the Recompiler, the performance uplift is night and day to say the least. Keep your eyes peeled on our YouTube channel for further updates!

Here’s a sneak peak at one of many gems that benefit from this improvement:

Closing Words

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. RPCS3 now has two full-time coders that greatly benefit from the continued support of over 800 generous patrons.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, MarioSonic2987, GalCiv, elad335 and Digitaldude555.

In addition to the following report, further details of Nekotekina and kd-11’s work during January and upcoming contributions can be found in their weekly reports on Patreon. This month’s Patreon reports are:

Status update from Nekotekina (2019-01-01)
Status update from kd-11 (2019-01-02)
Status update from kd-11 (2019-01-27)

Table of Contents

Major Improvements
Games
Other Improvements
Conclusion

Over at our forums, a few user and moderators have come together to acquire and test titles that have not been tested recently. Our developers have also been hard at work debugging niche issues that seem to prevent a few titles from progressing ingame. The results of their targeted efforts gives us veracious compatibility statistics from which we can see a big decrease in Intro and Loadable categories and a corresponding increase to Playable and Ingame categories.

On Git statistics, there have been 8179 lines of code added and 5073 removed through 36 pull requests by 9 authors.

Major RPCS3 Improvements

Before we get started, here’s a video showcasing some of the major improvements made to RPCS3 from December 2018 to January 2019:

Multithreaded SPU LLVM Compilation (#5586)

In January, Nekotekina implemented multithreaded SPU cache compilation during startup as well as runtime. This greatly improved startup times when using the SPU LLVM recompiler, especially for CPUs with high cores/thread counts. In Red Dead Redemption, a title known for having an excessive number of SPU programs, the compile time of about 75,000 objects on an Ryzen R7 1700 @3.9GHz went from 12 minutes 25 seconds to just 1 minute 34 seconds! Users can expect much faster startup times when using the SPU LLVM recompiler which was previously a big complaint with the experimental compiler. This improvement should also benefit SPU ASMJIT recompiler although the improvements would manifest as more stable framerate as opposed to an increase in FPS.

To implement the above feature, Nekotekina also improved adjacent logic such as SPU ubertrampolines and refactor the spu_runtime class. SPU ubertrampolines are a simple primitive performing a binary search over multiple program candidates in order to select the right one without knowing which one is currently loaded into memory. While the output was generated effectively, even with the SPU ASMJIT recompiler the output was simply not generated fast enough to meet RPCS3’s needs. Hence, Nekotekina implemented SPU ubertrampolines generation using raw assembly. Also, spu_runtime was refactored into a common class unifying the code used for both the ASMJIT and LLVM recompilers in this regard.

FBO improvements (#5427, #5565)

In December 2018, kd-11 had aimed to merge two major sets of changes to the RSX emulation. The first being improvements to the framebuffer object (FBO) management and the second being improvements to double assignments, conditional execution and shader inputs which was dubbed GT Fixes after the major series that benefited from it. While we’ve covered GT Fixes in last month’s report, this month we’ll be going over the FBO improvements which consisted of many improvements, the primary two major components being the removal of unnecessary framebuffer attachment for attachments that are likely unused and the rewrite of memory inheritance transfers.

The first improvement reorganised the framebuffer object management to avoid touching memory (a technique used to cheaply track memory usage) if a command was set up such that the data would not have changed. This allows RPCS3 to avoid generating framebuffer attachments for attachments that are likely unused and prevents memory corruption when Write Color Buffers/Write Depth Buffers is not used since we rely on attachment memory tags to watch for events. Previously, these corruptions affected multiple games such as Skate 3 where performance would abnormally drop below 1FPS while ingame and crash the emulator when entering skater edit screen.

Soon after, kd-11 rewrote the memory inheritance transfers to implicitly guarantee correct results even when Strict Rendering Mode disabled by invoking a memory barrier when actively reading from an unsynchronized texture. This simplified the memory transfer operations and allowed many games that currently require Strict Rendering Mode to display accurate graphics without the option enabled. Other notable improvements were the fix to the texture cache deadlock due to an incorrect internal size computations and increase in maximum number of compute invocations when using the Vulkan renderer from 120 to 1024. This was a result of manual benchmarking of various games demonstrating that it is completely possible to trigger a very high number of readback operations during loading screens.

For those of you wondering why this pull request was not merged in December 2018 itself, as part of kd-11’s original plan, the FBO improvements was slated to precede GT Fixes. However, while these changes did fix a variety of issues across a plethora of games, it also caused a large number of games to regression in multiple ways causing kd-11 to undertake a deeper analysis that what was initially envisioned. The main cause of these regressions were due to the large number of permutations possible on the framebuffer setup. This resulted in over 20 days of back and forth between kd-11 and our community of testers spiraling into a cycle of reporting and squashing bugs. Thanks to the continuous testing of almost every game and kd-11’s resilience to fix all the issues reported, FBO improvements became the most commented pull request in RPCS3’s history!

However, all efforts paid of as a plethora of games benefitted with improved graphics or fixed crashes. MotorStorm and MotorStorm Pacific Rift/3D Rift no longer crash frequently, Tekken Tag Tournament 2 shows accurate colors on clothes, Midnight Club: Los Angeles now renders rain effects, Saints Row IV now displays graphics when using the OpenGL renderer, Ratchet & Clank: QForce no longer suffers from the ghosting effect, Saint Seiya: Brave Soldiers has fixed depth buffer and much more. While the below images are a small selection of improvements, check out the improvements video linked above to see the improvements in many other titles!

Not one to slow down, kd-11 quickly began working on a follow-up pull request with various fixes for issues that were identified from the FBO improvements. These fixes included the rewrite of the index buffer base offsets to preserve the use of u16 index buffers for consistency and reduced memory footprint, a partial reversal of changes made in the FBO improvements affecting asynchronous shaders compilation, fixed red-blue color inversion when using the OpenGL renderer, avoiding a potential deadlock in RSX FIFO control, improving alpha-to-coverage transparency with a better approximation technique and rewriting the vertex attribute divisor logic.

These changes targeted various components of the RSX emulation and the resulting synergy was further improvement to graphical rendering and color accuracy. Midnight Club: Los Angeles saw tree shadows rendering for the first time and an improvement to foliage while MotorStorm games now display tire tracks accurately.

Audio Buffering & Backend Improvements (#5456)

When it comes to audio, RPCS3 has official support for 4 backends. On Windows, XAudio2 is available while ALSA and PulseAudio is available on Linux. Finally, OpenAL is a cross-platform audio backend that works on both Operating Systems. While the first three backends saw improvements overtime, the OpenAL backend was largely ignored and consequently was deprecated to a Do Not Use category (much like DirectX 12). One of our contributors, ruipin, evaluated the current audio setup and the various issues plaguing it and decided to undertake a large-scale refactoring of how audio is handled by RPCS3.

Issues relating to audio always posed a challenge to debug as many issues faced by users were the result of insufficient CPU resources. Since threads relating to RSX, SPU and PPU have higher priority, audio threads are delayed during bottlenecks leading to crackling or complete loss of sound. So in cases where the games couldn’t hit the target FPS, audio issues were prevalent even when gameplay was tolerable. However, titles like NieR had stuttery audio even when running at full speed on an i9 9900K. This revealed a deeper issue of games not being able to deliver audio in a timely fashion even when sufficient hardware resources were available. The PlayStation 3 provides audio in fixed intervals of 5.3 milliseconds but due to various factors, games could take longer than this once in a while, causing noticeable crackle and stutter.

To address these issues, ruipin implemented two new features, audio buffering and dynamic audio period. Together, these features aim to avoid intermittent stutter in games and provide users a smoother gameplay. To better understand these features let’s take a look at what each does.

Audio buffering is pretty straightforward affair with the audio being buffered into memory and played with a configurable delay (default being 100 milliseconds). However, dynamic audio period is where the magic truly happens. With it enabled, if the audio logic detects that the game is falling behind on audio mixing, it will dynamically increase the audio period to give the game some extra time. On the other hand, if the game seems to have audio ready earlier than necessary, the audio logic will dynamically shrink the audio period in order to maintain the average audio buffer duration and consequently ensure synchronization. This can continue with (almost) no stutter as long as the buffer does not run out. The below clip showcases the benefits of these changes:

However, ruipin went a step further and also introduced a time stretching algorithm which can be used to smooth out the crackles or stutter if the buffer fill level is falling dangerously low. This is meant to smoothen out rare stutter caused due any situation such as a background task kicking off or a loading screen using more CPU resources that usual. However, there is no pitch correction with this algorithm and hence if a game constantly has issues maintaining a full audio buffer (common on low thread count CPUs), this option will cause a huge reduction in audio quality. For this reason, this option is disabled by default.

On the other hand, audio buffering and dynamic audio period do not have such drawbacks and are enabled by default (on Windows) for all users to enjoy. However, these changes are not intended to be an end-all-be-all solution that magically fix audio in games that do not run at full speed. Your CPU needs to be fast enough to drive a full audio buffer every 5.3 milliseconds (on average).

Apart from these features, the OpenAL backend was also revived and brought up to the same quality levels as the XAudio2 backend. The entire cellAudio thread was also extensively refactored for easier maintenance and improvement in the future. All audio backends now automatically expose a list of capabilities so that the cellAudio algorithm can decide how to drive audio depending on what is supported. While audio buffering is enabled by default on XAudio2 and OpenAL backends (with a buffer duration of 100 milliseconds), PulseAudio and ALSA do not currently support these new features. Since the ALSA backend is the default option on Linux, users are required to enable the OpenAL backend to take advantage of these new features. Finally, Megamouse jumped in and added all relevant options to the GUI for the features mentioned above. You can find these options under the Audio tab of the Settings menu.

Game Dialog and Native UI Improvements (#5465, #5497, #5536, #5537, #5566)

In January, kd-11 announced that he would implement an On-Screen Keyboard (OSK) directly into the Native UI to allow users to enter text using the controller itself and not reach out for an actual Keyboard. However, a number of issues were identified with the existing Qt OSK dialog and Native UI that posed a challenge to the native On-Screen Keyboard implementation. Fortunately, our resident Quality-of-Life developer, Megamouse, decided to fix these issues to allow for a seamless integration of kd-11’s native OSK implementation.

The first set of improvements targeted bugs across various dialog systems in RPCS3 (Qt and Native UI):

• A regression from the December’s controller configuration improvements that caused games to sometimes freeze which opening or closing Native UI interfaces was fixed.
• Next, titles such as Diablo 3 would assume that the dialog was accepted, even when it was actually cancelled. To address this, Megamouse ran test on the PlayStation 3 to determine how the original hardware handled cancellations and implemented the same accurately in RPCS3.
• After this, there was an issue where the game would continue to recognise input from the controller even when the text input dialog was open. With the Qt implementation, this did not pose much of an issue as all text was fed with a keyboard while input to the game was given with a controller. But once the native OSK is implemented, games incorrectly recognising input that was meant for the OSK would cause unwanted issues for users. To address this, proper system dialog input interception was implemented to discard any input made to the game while a system dialog is active.
• A missing check for empty strings was implemented which fixed a crash when users tried to confirm an input dialog without entering any text. Some games such as Diablo 3 will now use this check to identify whether the string is empty and notify the user accordingly after closing the dialog.
• Finally, few constants and checks were added to improve emulation accuracy.

Previously, Godzilla would freeze after the language confirmation dialog but thanks to the above improvements, this title goes ingame once again!

The above improvements fixed issues affecting multiple titles such as Lair, South Park: The Stick of Truth, Sengoku Basara 4, Diablo 3 and Godzilla. However, a number of other titles such as Skate 3 and NieR began to crash unexpectedly when interacting with game dialogs. Investigating these regressions, Megamouse discovered that these issues were not true regression but were in fact various unimplemented features that were brought to the forefront thanks to the improved accuracy of the message dialog system. Once these missing permutations were pinned down, the features required to address “regressions” were summarily implemented.

Support for displaying the game’s message to the Qt OSK dialog (and future OSK interface) was also implemented

To aid in debugging and facilitating further improvements, Megamouse also improved the error code logging for all system dialogs and separated the existing OSK dialog code into its own class. This will also help the smooth implementation of the Native UI On-Screen Keyboard. Finally, the HLE implementation of OSK dialog was improved by reimplementing state checks (open, closed, aborted) and making sure that data isn’t written to unallocated memory whilst passing the user input to the game’s buffer. This was a longstanding issue that was finally addressed issues in titles such as Class of Heroes 2G.

Games

Skate 3

The third installment in the Skate franchise, Skate 3, is now fully playable! This title previously suffered from major slowdowns in the free skate mode but thanks to improvements made by kd-11 to the framebuffer management, this issue has now been resolved. Check out the performance of this title on RPCS3 in the video below:

Dragon Ball Z: Battle of Z

One of the last titles in the Dragon Ball series to grace the PlayStation 3, Dragon Ball Z: Battle of Z has now become playable! This title also benefited from kd-11’s framebuffer management improvements which fixed a regression that caused the game to crash at the title screen.

Ni no Kuni: Wrath of the White Witch

Ni no Kuni: Wrath of the White Witch is now completely Playable on RPCS3 at a good framerate even on mid-range systems thanks to recent improvements from Nekotekina and Eladash. While Ni no Kuni has run well on good systems for a long time, the recent graphical, performance and stability improvements have finally pushed it over the edge into the playable category.

No More Heroes: Heroes’ Paradise

While this console exclusive title had good performance and graphics, it suffered from a bug preventing users from being able save their progress. Thanks to the recent improvements to save data handling in RPCS3, this issue has been fixed and No More Heroes: Heroes’ Paradise is now fully playable! A user from our forum, MilkManEX, managed to finish the game from start to end on RPCS3 without any major issues.

Onechambara Z: Kagura with NoNoNo!

Did you have your PlayStation 3 set-up in the living room? Were you denied the chance to hack down zombies whilst wearing a bikini because of this? Well then, your wait is over. This month, Onechambara Z: Kagura with NoNoNo! was found to progress ingame! While the game has stable performance and graphics, it does suffer from crashes during boss battles keeping it from being playable.

E.X. Troopers

As is customary, here’s another weeb school/mecha/action/shooter title that is now fully playable on RPCS3.

Ragnarok Odyssey Ace

This JRPG previously suffered from significant graphical glitches keeping it from being playable. However, with recent improvements to RSX emulation, these glitches have been fixed allowing this title to be fully playable.

WWE All Stars

This critically acclaimed WWE spin-off is now full playable! Previously this title would crash on the loading screen before going ingame. With the recent stability improvements these issues have now been fixed.

Absolute Supercars & Ferrari titles

Thanks to the RSX vertex base type improvements by eladash, multiple racing titles from Eutechnyx studio (Absolute Supercars, Ferrari Challenge: Trofeo Pirelli and Ferrari: The Race Experience) went ingame for the first time this month. However, these titles still suffer from few graphical issues and low performance keeping it from being playable.

Elevator Action Deluxe

Thanks to kd-11’s improvements, the last remaining bug affecting the HUD was fixed, allowing the game to be fully playable now!

Before	After

Vampire Rain: Altered Species

Thanks to the recent stability improvements made to the emulator, this survival horror title went from Loadable to Ingame!

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details, as well as the code changes themselves. To see the complete list of pull requests directly on GitHub, click here.

Nekotekina

5453 – Changed the cache location to a dedicated folder. For the PPU module cache a new hash-based location was implemented similar to SPU cache. Also, the structure was changed to decrease size of each module resulting in a bump in the PPU cache version; Fixed a minor race condition in cellVdec and other misc. improvements;

5572 – Changed audio backend priority on Linux to ALSA. Also, fixed few cases of error spam with cellMsgDialog on CELL_OK;

5586 – Implemented support for parallel compilation of SPU code, both at startup and runtime; Removed the obsolete “SPU Shared Runtime” option; Refactored spu_runtime into a common class for both ASMJIT and LLVM recompilers; Implemented SPU ubertrampoline generation in raw assembly for the SPU LLVM recompiler. See coverage in major improvements here;

5599 – Improved compilation speed and performance in a few games when using the SPU ASMJIT recompiler;

5614 – Fixed a rare crash in cellSaveData by retrying to move directories on FILE_ACCESS_ERROR.

kd-11

5509 – Updated Vulkan descriptor pool init sizes to prevent descriptor pools from running out of resources with only 25% occupancy; VK_WHOLE_SIZE will not be used for the null resource bind since most drivers have uniformBufferRange limits that are much smaller; Clamps swapchain resources to allowable surface extents;

5427 – Reorganized framebuffer object management to avoid touching memory if a command was set up in a manner where the data would not have changed. See coverage in major improvements here;

5565 – Rewrote the index buffer base offsets to preserve the use of u16 index buffers for consistency and reduced memory footprint; Partial revert of a change in the above pull request affecting asynchronous shaders; Fixed red-blue color inversion when using OpenGL; Avoid potential deadlock in RSX FIFO control; Reimplemented alpha-to-coverage transparency with a better approximation technique; Rewrote vertex attribute divisor logic. See coverage in major improvements here;

5612 – Set preference for the slower FIFO mode for VSync instead of Mailbox mode as the latter is broken on recent Nvidia drivers when using Vulkan on Windows. Also implemented fence timeouts to detect dead renderers if a crash happens in submit before vkGetFenceStatus can return relevant data.

5618 – Fixed a regression from the above pull request where game window couldn’t enter fullscreen mode on Linux when using Vulkan.

elad335

5540 – Allowed invalid NV4097_NOTIFY context to pass execution that addressed an incorrect exception thrown in Ni no Kuni: Wrath of the White Witch!

5522 – Allowed more than one SPU MFC list to execute in a tag group when one list is stalled which now supports multiple list stalling and unstalling in a tag group. This improvement now matches RPCS3’s support for MFC list stalling to that of an actual PlayStation 3;

5435 – Improved LR event handling of SPU GETLLAR, PUTLLUC and PUTLLC transactions resulting in higher accuracy. Also, reduced lifetime of vm::writer_lock in PUTLLC transactions for non-TSX CPUs, replaced cache line spinlocks with a new passive range lock in MFC PUT DMA transfers and other improvements which resulted in a significant performance increase in multiple titles for CPUs without TSX;

5568 – Patch ppu main thread priority in cases where the PPU main thread priority specified by the executable file is invalid the default value is used like an actual PlayStation 3. With this improvement, White Album 2 went from Loadable to Ingame!

5585 – Allowed vertex base type 0 to pass execution which was not allowed previously due to its unknown behaviour. With this mode correctly implemented, games such as PixelJunk Eden, Absolute Supercars, Ferrari Challenge: Trofeo Pirelli and Ferrari: The Race Experience were able to reach ingame!

5579 – Fixed spurious failure of try_rlock/wait methods in lv2 semaphore by restarting the atomic loop and waiting for condition confirmation instead of aborting prematurely on failed CMPXCHG;

5595 – Improved accuracy by making sys_ppu_thread_isjoinable return an error with the proper error code when trying to join the main thread;

5605 – Fixed potential crash in begin_occlusion_query() while closing games;

5502 – Improved compatibility of DECR mode by fixing the 512MB memory area allocation by sys_memory in 1MB pages mode. This fixed WSC Real 09: World Snooker Championship and allowed the game to progress ingame! However, further improvements are necessary for the DECR mode allocations to be fully accurate.

Megamouse

5465 – Part 1 of the game dialog fixes and native UI improvements. See coverage in major improvements here;

5497 – Part 2 of the game dialog fixes and native UI improvements. See coverage in major improvements here;

5536 – Part 3 of the game dialog fixes and native UI improvements. See coverage in major improvements here;

5537 – Part 4 of the game dialog fixes and native UI improvements. See coverage in major improvements here;

5566 – Final part of the game dialog fixes and native UI improvements. See coverage in major improvements here;

5493 – Implemented an “Exit game?” dialog that lets you abort the exit process and also warns about loss of progress. A similar dialog will also spawn when you boot a game while another game is running. The dialogs can be disabled via “Don’t show again” checkbox and re-enabled in the GUI settings. All other info dialog settings were added to the same location as well;

5491 – Improved the builtin debugger to now only allow hex values with length 8 in the “Go to address” pop-up. Also improved handling of the “No Thread” option to prevent some crashes;

5531 – Fixed the macOS Travis builds and upgrades Travis to use Xcode 10.1. Also, homebrew packages for macOS are now installed using travis.yml instead of build-mac.bash;

5528 – Made dock-widget title bars optional by adding a new checkbox to Menu & View;

5529 – Fixed an issue in cellGameGetSizeKb during installation of a game when a directory wasn’t found by returning size=0 instead of an error. Allowed Tom Clancy’s H.A.W.X 2 to progress from Loadable to Intro;

5548 – Added error code log messages to cellSaveData in order to improve debugging of issues relating to game saves;

5574 – Improved the game list by using the the correct patch version for PS3 disc games. Previously, the game list only showed 1.0 for all disc games and the actual version was only found in game data entries;

5588 – Fixed an obscure error in the shader compilation dialog that displayed the wrong progress bar status. Also, set the error for the notorious underrun message to log level warning;

5587 – Changed the default location for all per-game custom configurations to config/custom_configs/ and modified their naming format to include the game ID as well such as config_ABCD12345.yml. This feature is backwards compatible with the old locations and old config files will still be considered but the new config files will take precedence over the old ones. Also, added the option to boot games either with their custom configurations or with the global configurations.

ruipin

5456 – Refactored the audio backends and introduced audio buffering and time stretching which helps avoid intermittent stutter (currently only for XAudio2 and OpenAL). See coverage in major improvements here;

5539 – Fixed regressions caused in the above pull request.

GalCiv

5461 – Fixed issues with initial trophy set-up by accurately simulating the PlayStation 3’s behavior through a mix of forced granularity and timeouts for games that don’t check the progress while processing the trophy entries. This improved compatibility for games such as Diablo 3, Odin Sphere and Mahjong * Dream C Club;

5535 – Reverted a faulty section in the cellGameGetSizeKb changes made in PR 5529.

AniLeo

5525 – Updated the Readme file to show Qt5Qml as a dependency when using Arch Linux;

5619 – Bumped the RPCS3 version to 0.0.6.

MSuih

5558 – Fixed a bug where the GUI option for removal of shader cache, PPU cache and SPU cache incorrectly pointed to the old cache location.

maximilian578

5607 – Added appropriate styling to QDoubleSpinBox when using the Kuroi (Dark) and YoRHa stylesheets.

Closing Words

We hope you liked this report and look forward to the next! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

We’re always looking for dedicated writers to help us write these reports. If you have the skill, time and are willing to help, please apply here. Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

This report was written by HerrHulaHoop, DigitalDude555, Megamouse, eladash and GalCiv.

December has been an absolute whirlwind for RPCS3 with improvements made to various parts of the emulator. Nekotekina greatly improved performance with the introduction of Approximate xfloat while kd-11 fixed a wide variety of graphical issues affecting multiple AAA titles. Apart from our full-time developers, our regular contributors also made a plethora of improvements to the texture cache predictor, cellCamera emulation, controller/mouse support, GUI and tons of bug fixes. We’ve tried to detail as many as we possibly could in this report. So without further ado, let get into it!

In addition to the following report, further details about Nekotekina and kd-11’s work during December and their upcoming contributions can be found in their weekly reports on Patreon. December’s Patreon reports were:

Status update from kd-11 (2018-12-10)

Table of Contents

Major Improvements
Games
Other Improvements
Upcoming
Conclusion

This month saw improvements to various aspects of the emulator and the compatibility list is no exception. At the start of 2018, the compatibility list was revamped to allow us to merge multiple game IDs for the same game into one single entry. This helped reduce duplication of entries and enhance the quality of the statistics we provide. While we were able to merge IDs of different regions together, due to the way the compatibility list database was structured, titles with multiple IDs from the same region could not be merged and hence were shown as separate entries. Not wanting to leave the job half done, AniLeo refactored the compatibility list once again this month to now allow us to merge IDs from the same region as well!

With this change, we can eliminate unnecessary duplication entirely and significantly improve the quality of our statistics to properly represent the amazing progress made with the emulator. If you take a look at the metrics for this month, you will notice that the number of games has decreased across all categories. This is solely due to the above mentioned revamp of the compatibility list. However, if you take a look at the compatibility history page, we can see that the Playable category has increased by over 15 titles and Ingame category has increase by over 30 new titles. The nothing category also reduced by one, bringing the total to just 3 titles!

On Git statistics, there have been 4,802 lines of code added and 2,530 removed through 105 commits by 18 authors.

Major RPCS3 Improvements

GT Fixes (#5436)

In December, kd-11 fixed numerous issues in a large variety of games. For starters, he laid the foundation for fixing display output issues, where rendering was squeezed into one corner of the screen. Shader decompilation was also improved in several ways and the fixes that stood out the most were correcting how double assignments (like x = y = z) are resolved, the reimplementation/ refinement of conditional execution so as to avoid data corruption, and the separation of depth and stencil data for shader inputs, were they to be accessed simultaneously. While the first of the three is pretty self-explanatory, the latter two need some further explanation, so let’s do precisely that!

First, let’s see what conditional executions needed a fix for! Say you had three pixels (a, b and c), and you wanted to modify ‘a’ pixel’s color so that a = b * c (in color values). Since a pixel’s color is determined by the values of their channels (alpha, red, green, blue) this means doing this multiplication for every channel, one-by-one (so, not following the official notation, a.red = b.red * c.red, a.blue = b.blue * c.blue and so on). You, being unhappy with this result, decided then that you wanted, say, the green channel excluded from this process. You can do that by relying on a so-called choice condition register, in the following fashion: you put a pixel value in it, and you set each of its channels’ value to 0 or 1. If a channel is 0 in the pixel stored inside the choice condition register, it means that exact channel will be excluded from the process laid out above. Knowing this, you set the green channel’s value to 0, set 1 for all the others, and you go ahead with the execution. You can now check for each channel’s value in the register before executing the original command for that given channel (so before you do a.green = b.green * c.green you test against cc.green, where cc is the choice condition register).

This is nice and all, but let’s say you wanted to go even further than just multiplication. In a more abstract sense, this means putting the ‘a’ pixel into some function, that function does something with it, and you read the resulting value back to ‘a’ for each channel. Anyone who ever did software development will now raise their heads up instantly: since we are assigning the result of function(a) back to ‘a’ channel-by-channel, and not at once, we repeat the function again and again, but use the channel values that have already been altered with each new run instead of the original ones. In hindsight, this is an obvious data hazard and was easily fixed by moving the function execution before the assignment, instead of repeating it for every channel. Of course on GPUs, this is quite a bit more elegant: a = lerp(a, function(a), cc);. This change affected games where corrupted color was perceptible:

Now that we got this one down, let’s see what the separation of depth and stencil data was needed for. Due to API limitations, on PC, you cannot read depth and stencil values simultaneously from within the same shader. On the PS3 however, this is not a problem: you can do so by casting (~forcing) the given D24S8 texture (depth data of 24 bits + stencil data of 8 bits = 32 bits) into an ARGB8 texture (alpha, red, green, blue, with each being 8 bits, so in total = 32 bits), and then reconstruct the original depth and stencil values inside the shader, using the individual channels.

In order to retain the original form of the shaders, we previously tried to mimic this process as follows: we casted the depth data into color bits (R,G,B), and stubbed the stencil value into being 1 always, residing in the transparency channel (A). This worked surprisingly well for the longest time, but as it turns out, games like God of War or titles from the Gran Turismo series used the stencil part to encode data required for proper font rendering.

To fix this, kd-11 gave up on retaining the original shader’s form and now passes the depth and stencil data as separate textures. The only caveat is, that instead of having 16 inputs at max, shaders can now have up to 32 inputs when all the input textures are needed to be split – something that isn’t possible on all drivers. In turn, proper font rendering (among others) is now possible. This improvement benefitted a multitude of titles including titles from the Gran Turismo series that saw a distinct uplift to graphical output:

Approximate xfloat (#5430)

Back in August 2018, Nekotekina implemented accurate handling of extended floats when using the SPU LLVM recompiler. As previously detailed in that month’s progress report, this brought massive improvements to accuracy but with a significant decrease in performance. Due to the negative impact in performance, Accurate xfloat was retained as a toggleable option that was disabled by default. A large number of titles compulsorily required Accurate xfloat to be enabled when using the SPU LLVM recompiler as they would otherwise suffer from broken physics, missing geometry, incorrect models, etc.

While Accurate xfloat provided the most effective solution, its performance hit was significant enough that only high-end CPUs could utilise the option and still maintain playable performance. Users were left with the choice to either upgrade their CPU to enjoy full accuracy or stick with an unplayable experience. There was a clear need to have a midway solution that allowed for better accuracy without degrading performance. This was also clearly evident as multiple hacky builds of RPCS3 began to popup that simply carried over the quick workaround that kd-11 had previously implemented to the SPU ASMJIT recompiler.

To fill this void, Nekotekina implemented Approximate xfloat which effectively adapts kd-11’s workaround for certain instructions while forcing accurate handling for others. This resulted in huge performance gains in games that previously required Accurate xfloat such as God of War 3, Ninja Gaiden 2, Sly Cooper Thieves in Time, Ni No Kuni, Uncharted, Macross, Wipeout HD, Heavy Rain, Dante’s Inferno and Journey. Using this options allows users to reap most of the benefits from Accurate xfloat without any performance reduction. Consequently, this option is now enabled by default when using the SPU LLVM recompiler.

Here’s a comparative look of the performance improvement in Sly Cooper: Thieves in Time running on midrange hardware

With the implementation of Approximate xfloat, the SPU LLVM recompiler now boasts higher accuracy and performance compared to the SPU ASMJIT recompiler by default. With further optimisations and fixes, the SPU LLVM recompiler is set to become the definitive backend to use in RPCS3!

God of War 3 sees a significant improvement to performance now when using SPU LLVM recompiler

Texture Cache Predictor Rewrite (#5276)

In our September report, we covered ruipin’s various texture cache related changes in great detail, but most importantly, we also introduced the concepts of texture locking and hard faults. Since the aforementioned hard faults are a key concept of this change, here’s a quick refresher: when the Cell tries to read from a read-protected texture region, we first have to make sure that said region is up to date. If that isn’t true, we need to halt emulation and flush the texture from VRAM to RAM to resynchronise – and that is what’s called a hard fault. Its downside is sadly obvious: if we need to halt the emulation process to wait for such texture uploads, performance will suffer, and suffer greatly.

To alleviate this, there was already a predictor algorithm in place. As its name suggests, it’s job was to predict if a game will need a certain texture ready, so that by the time the game actually asks for it, we’ll already have it uploaded into the RAM. Unfortunately however, since predictor data before the rewrite was stored in each texture’s data structures, the heavy reuse and reset of those structures cause by the rewrite made the predictor plenty less reliable. Adding insult to injury, the original predictor wasn’t terribly sophisticated to begin with, so with September’s changes in place, its effectiveness by and large reached a minimum.

This is where ruipin’s predictor rewrite came into play. To restore the predictor’s hit rate, he redesigned its heuristic from scratch. Textures are now kept track of in a dictionary via so-called “predictor keys”, which are basically hashes of batched-together texture metadata of each texture in the cache. This enables the predictor to now correctly react whenever it comes across a previously seen texture, and know more about its usage statistics. In addition to this, various fine-tuned adjustments were made for edge cases, such as overly frequent flushings of a given texture, and more. This change not only restored the predictor’s efficiency, but even increased it beyond that of the original, leading to way less hard faults in games such as TLOU or R&C ToD, and virtually no more mispredictions (pointless flushes).

With the new texture cache predictor, you can see the number of hard faults reduce to just a single hard fault with a corresponding increase in performance

Controller Improvements and improved mouse support (#5425, #5426, #5443, #5468)

Every month, our inhouse Quality-of-Life developer Megamouse tries to improve your experience using RPCS3 and December was no exception. The controller handler was restructured to allow the smooth implementation of some much needed features. These improvements benefitted multiple input devices such as the DualShock 4, XInput devices and computer mouse.

The first improvement was the restructuring of the pad implementation to benefit all controller backends. As users of RPCS3 would already know, the emulator does not allow its settings to be changed while a game is running. For any modified settings to take effect, the game would have to be restarted and this was also true for the Pad settings. This soon turned into an inconvenience for many users who had to restart the game simply to adjust deadzones or vibration intensity. Some users also needed to restart the game if their controller accidentally disconnected during gameplay.

To alleviate these issues, the Controller handling was improved to allow for live detection of controller status and application of configurations made in real-time. Users can now change and save all pad settings while being in-game. It is also possible to switch from one controller backend to another seamlessly, for e.g., switching from Xbox One controller to Keyboard+Mouse. Once users have made the necessary changes, they can simply hit save in the Pad configuration dialog and the new settings will be immediately applied to the game. Along with these improvements, the connection status of controllers will now show up in the log so users will be able to clearly see whether or not their controllers were detected by the emulator when connected. In the event that the controller is disconnected, no remapping will be allowed to preserve the saved settings for such controller. Any button reassignment that was being made during such disconnection will also not be saved. This prevents accidental errors when the controller’s connection is weak.

The log will now display when controllers are disconnected and reconnected	When the controller is disconnected, no remapping is allowed and all options will be greyed out

The second feature implemented was improved support for binding Mouse to analog sticks. While this feature was implemented in August 2018, the implementation had some drawbacks. The primary deficiency being that native deadzones offsets weren’t saved and would have to be set every time the game was booted. The second being that the process of modifying the offsets was primary driven through log messages making it that much more to set-up. Fortunately, all of these drawbacks were addressed this month. Mouse parameters set now save along with the Keyboard config profiles so users no longer have to reapply the settings every time. The Pad configuration dialog now also displays neat options to easily modify the mouse acceleration and deadzones thresholds.

Options to modify mouse acceleration and deadzones have now been added to the Pad configuration dialog. For reference, this is how the Pad configuration dialog would look when the right analog is bound to the Mouse.

Apart from UI improvements, stick interpolation was also added to help further improve the experience when using the Keyboard. When using the Keyboard, input for analog sticks are restricted to only 4 keys (e.g. W A S D) and consequently only 4 directions. However, when using a regular controller, users have the benefit of moving in any direction that the analog stick resolution supports. Therefore, emulation of analog sticks with the Keyboard has an inherent disadvantage that could negatively affect a user’s experience. This is where stick interpolation comes into play. RPCS3 now smoothens the key input for the left and right analog sticks over time using a basic lerp (linear interpolation) per stick axis (4 axis in total, 2 for the left and 2 for the right).

By default the interpolation level is set at 1.00 (i.e. no lerp is applied). Lowering this value will make the analog stick smoother to move. This is especially useful when, for example, you want the character to turn normally instead of simply snapping into that direction. However, do remember that setting the interpolation value to 0.00 will result in no movement to be recognised from that analog stick.

Apart from these awesome updates, there were also some miscellaneous improvements made to other specific controller backends. Settings for the DualShock 4 was made more dynamic by using an index from 1-7 instead of HID-Device-IDs. In line with the above changes, the DualShock 4 backend was also improved to allow for seamlessly disconnecting and reconnecting during gameplay. For XInput controllers, support for the usage of a single controller for multiple players was also added. While this is a niche feature, users who requested for the same now have reason to celebrate!

Games

God of War 3

God of War 3 is one of the many titles that previously required Accurate xfloat enabled to fix physics issues when using SPU LLVM. Now, thanks to Nekotekina’s addition of Approximate xfloat, this title has seen a massive improvement to performance. Check out the video below to see just how much performance has improved:

El Shaddai: Ascension of the Metatron

Previously, this console exclusive title suffered from random crashes during gameplay that prevented it from being playable. Thanks to the recent stability improvements made to RPCS3, this title is now fully playable! A user from our forums (aptly named Testing) has finished this game on RPCS3 without any major issues.

WET

This month the action packed console exclusive made its way to Playable! With the improvements made to the emulator, the graphical issues and crashes that were previously plaguing this title have now been fixed. Do remember to enable Strict Rendering Mode to fix background flickering.

Motorstorm 3D Rift

This month, the PlayStation 3 exclusive Motorstorm 3D Rift was found to be ingame. Previously this title would crash after the menus. With this, all titles from the MotorStorm series reach ingame! While this game does have decent performance, it does suffer from few graphical glitches keeping it from being Playable.

ModNation Racers

Thanks to the implementation of DECR console emulation support by elad335, ModNation Racers now goes ingame! However, the game suffers from low performance and graphical issues preventing it from being properly playable.

Golden Axe: Beast Rider

Golden Axe: Beast Rider is yet another console exclusive that was found to progress ingame this month. Within a week of this discovery, user Bnd_ps3 from our forums managed to complete the game on RPCS3. However, from his experience, the game suffers still suffers from random freezes and graphical issues which prevent it from being categorised as Playable. Here’s some gameplay footage to see how this game currently performs:

cellCamera titles

Thanks to Megamouse fixing a faulty check that lead to inconsistencies in cellCamera, various titles that required the PlayStation Eye now progress further. Operation Creature Feature and The Trials of Topoq managed to reach intros whereas Aqua Vita went ingame! However, proper PlayStation Eye emulation would be necessary for these games to be categorised as Playable.

Everybody’s Golf 6 (Hot Shots Golf: World Invitational)

If you’re a sports fan and consider golf a sport, then we’ve got some great news for you! The latest outing from the Everybody’s Golf series to grace the PlayStation 3 now goes ingame. While the game looks visually gorgeous, performance is too low to consider this title as Playable.

NCAA March Madness 08

If you don’t consider golf a sport, worry not, this saw the classic NCAA March Madness 08 progress ingame for the first time. However, performance during gameplay is low even on high-end hardware. Further optimisations to the emulators is necessary to obtain Playable performance from this title.

Natsuiro High School: Seishun Hakusho

With the multiple accuracy improvements made to the emulator in the recent months, multiple games moved from Loadable directly into Ingame. One such title is Natsuiro High School: Seishun Hakusho. While the game generally has good performance and graphics, certain areas reduce the FPS signficantly, keeping this title from being Playable.

Mobile Suit Gundam: Side Stories

For all mecha anime fans out there, this month saw two such titles go ingame for the first time this month. The first is Mobile Suit Gundam: Side Stories which is a remastered collection of side stories from older generation titles. While this title is relatively stable, it does suffer from low performance and a few minor graphical issues.

Rinne no Lagrange: Kamogawa Dream Match

The second mecha title was the PS3 exclusive title Rinne no Lagrange which was found to be playable this month!

Other Improvements

There have been numerous other pull requests merged during the month that couldn’t make it to the Major Improvements section. We have collected a list of all such improvements here, and attached a brief overview to each. Make sure to check out the links provided for them if you are interested, as their GitHub pages usually uncover further details, as well as the code changes themselves. To see the complete list of pull requests directly on GitHub, click here.

Nekotekina

5400 – Minor improvement to exception handling in the RSX Thread by detaching VBlank and RSX Decompiler threads;

5414 – Minor improvements to cellSaveData to address the “Already exists” error;

5430 – Added an approximation of xfloat for SPU LLVM, which provides substantial improvements to games that previously required “Accurate xfloat” to work. See coverage in major improvements here.

kd-11

5391 – Fixed regressions caused by the FIFO processing rewrite PR 5315;

5436 – Fixed display output issues, shader decompilation issues, correction of how double assignments are resolved, reimplementation of conditional execution and separation of depth and stencil data for shader inputs. See coverage in major improvements here.

Megamouse

5393 – Fixed a minor inconsistency in the GUI that caused the “Play” button to display “Pause”;

5412 – Added the ability to customize the title of games present in the game list. The new name will be used for any search and sorting operation and can therefore be used to improve the game list management and customization. Also fixed an issue where you could mistakenly rename titles in the game grid;

5425 – Enabled the usage of one XInput controller for multiple players. Though the functionality was present in the emulator, this feature was not enabled or reported. See coverage in major improvements here;

5426 – Added the ability to make changes to the pad settings during gameplay. Users no longer have to restart the game for the changes to apply and the new changes will take effect spontaneously. See coverage in major improvements here;

5439 – Fixed a faulty check that lead to inconsistencies in cellCamera. This fixed issues with various games such as Aquatopia, Trials of Topoq and Operation Creature Feature and allowed them to progress further!

4908 – Correctly report game window sizes to the graphics backend with HiDPI screens. Windows users should use Qt’s HiDPI mode instead of Windows’ HiDPI mode. Please note that both modes aren’t perfect so this change might be reversed or left up to the user in the future;

5443 – Improved all controller backends by changing the index to 1-n instead of 0-n, adding connection status to the device names, disabling remapping when a controller is disconnected and immediately cancelling current remapping in case the controller is disconnected. Also, specific improvements were made to the DS4 handler by using an index from 1-7 instead of HID-Device-IDs which makes the DS4 settings more dynamic and made it possible to reconnect DS4 controllers. See coverage in major improvements here;

5464 – Fixed unreadable out-of-focus titles for selected games in the game grid by adjusting the concerned style sheets;

5474 – Fixed a mismatch of table header states and icon sizes that occured when the emulator crashed and therefore couldn’t save the header widths. This lead to wrong sizes of the first game list column and occurred frequently when stopping execution in Microsoft Visual Studio;

5476 – Updated outdated tooltips in the settings dialog, updated some setting names to make them more transparent to users, removed “Force Disable Anisotropic Filtering” from the UI dropdown to prevent incorrect usage of the option and moved the obsolete setting “GPU Texture Scaling” to the debug tab;

5468 – Added mouse movement and stick lerp settings that will be applied when choosing the Keyboard as the controller. Also, added the relevant settings to the pad settings dialog for easy modification. See coverage in major improvements here;

5484 – Added a warning log entry “Compiling PPU module: …” at the beginning of a PPU module compilation and a success log entry “Compiled PPU module: …” at the end of a PPU module compilation.

hcorion

5441 – Added a CMake switch to disable Discord Rich Presence entirely;

5445 – Removed the existing Linux distro whitelist and use the new automated stdc++ checker;

5479 – Deploy Linux binaries to GitHub releases. Now, Linux also has a build history on the website!

GalCiv (RipleyTom)

5384 – Fixed an issue during the maximization of the game window when using Vulkan with an Nvidia GPU where the render did not resize properly. The fix was to simply send the resize message to RSX on the last possible message;

5388 – Fixed a regression caused by the above PR where the render did not resize properly when using Vulkan with an AMD GPU;

5399 – Fixed a regression in cellSaveData that caused titles like Battle Princess Arcadias to crash on launch. This was fixed by adding more result handling from funcStat CB in savedata_op. This fix actually came from an investigation as to why some games started with no sound and you had to go into settings to increase the sound volume;

5304 – Fixed an error in cellFsGetFreeSize where previously RPCS3 used the full path to determine if free space was available when the PS3 only used the device (/dev_hdd0, etc.) in the path. This fix addressed issues in games like LittleBigPlanet, No More Heroes: Heroes’ Paradise and Moon Diver where the game would fail to create a save file due to supposed lack of space;

5450 – Improves trophy handling accuracy;

5457 – Fixed few issues in cellVideoOutConfigure that allowed Karaoke Revolution to progress further!

5475 – Fixed a race condition where the mutex is destroyed whilst an unlock is happening. The crash occurred due to an object being destroyed by a thread while it was still in use by another thread. This fixed the elusive crash in Demon’s Souls that only affected CPUs with low thread counts (4C4T).

elad335

5381 – Fixed few RSX image_in command bugs and implemented support for negative scaling. This fixed crashes or blackscreens in a number of titles such as Surf’s Up, NCAA March Madness 08, Mamorukun Curse and Demolition Inc. Games fixed by this fix need Write Color Buffer enabled to work correctly. Further work is necessary for this fix to be fully functional without Write Color Buffers. Also, the RSX capture tool was simplified by removing the memory map save in save state and instead opting to simply preallocate all memory;

5392 – Optimized the PUTLLC MFC command execution by removing 128 byte long redundant read and writes from the compiled code;

5363 – Fixed PPU link register (temporary one-layer return address saver when calling functions) by saving the address of the next instruction even if the jump condition failed;

5419 – Fixed a bug that caused input errors in the native UI. This was achieved by initializing the pad handlers before a game actually initializes them itself, just like a real PS3 also has input before the game starts. This fix also made it easier to follow up with the other pad config changes described above;

5404 – Fixed remaining issues and regressions with RSX FIFO emulation, making some games more stable such as Red Dead Redemption and Ni No Kuni. Also improves RSX debugging in the emulator;

5440 – Improved error handling for the “Null” camera option so that certain games can work properly when no camera is selected;

5334 – Fixed SPU group status after all threads exited individually when not joining the SPU group and simplified SPU group status set after termination;

5459 – Fixed an inaccuracy in sys_semaphore_post syscall where an error code should be returned instead of success status when passing the parameter count = 0;

5466 – Fixed the capture of FIFO commands with a memory state and also improved precision of the conditional rendering checking;

5357 – Implemented support for DECR console (debug unit) emulation and corrected the available memory size reported to games. The DECR console has 256MB of additional RAM in comparison to retail consoles. Since memory status reports to games are not fully accurate, using the Debug mode addresses issues with memory limitations while also making some games expose debug functionality such as printing debug TTY messages and displaying debug menus. Using this mode, Pirates Of The Caribbean: At World’s End now reaches Intro!

ruipin

5276 – Fixed various regressions with the texture cache caused by PR (5115). In addition, it implements a proper texture cache predictor, that attempts to predict when textures will need to be copied from GPU memory to host RAM, and thus can queue up that operation in advance, avoiding a hard fault and improving performance. See coverage in major improvements here.

scribam

5386 – Fixed Marvel Pinball by adding the error check SCE_NP_ERROR_ID_NOT_FOUND to sceNpBasic entry functions;

5451 – Fixed an issue with the time counter in Deathmatch Village where the duration would be stuck at 30 minutes.

MSuih

5382 – Added game list category filters for PS1, PS2 & PSP games;

5411 – Implemented an option to check the cache size (dev_hdd1/cache) and delete the oldest folders until it’s less than a configured threshold. Users can enable this feature and customize the threshold in the System tab;

5448 – Improves detection of Hyper-threading with Intel processors on Windows.

JohnHolmesII

5360 – Cleaned up the Readme file and moved uncritical information to the RPCS3 wiki;

5473 – Fixed building RPCS3 with musl libc.

jbeich

5467 – Fixed an exception that caused games to crash on FreeBSD.

isshininu

5437 – Addressed an issue where the trophy data for a few games (e.g., The Evil Within) is not correctly loaded trophy data.

hrkrx

5433 – Added libevdev-dev as requirement for Debian & Ubuntu to prevent user errors.

janisozaur

5379 – Improve path to discord-rpc submodule when building with non-MSVC compilers;

5387 – Updated hidapi submodule to address mingw compilation issues.

guigzzz

5385 – Added ignore = dirty to all submodules to make git operations faster.

NicknineTheEagle

5370 – Fixed paths to PARAM.SFO and game icons for C00 games.

vit9696

5367 – Added support for macOS bundling for binary distribution.

fobb2

5472 – Fixed typos in a few tooltips.

Upcoming

Are you still here? Great! ‘Cause there’s way more for us to cover!

January was definitely not a boring way to start the year and was packed with a ton of updates. Not only did we finally have a multithreaded SPU cache compilation path added thanks to Nekotekina (much to the relief of our testers), but we also had native OSK (on-screen keyboard) support added as well as a much welcome audio improvement done over the course of the month. And who knows, maybe many of our users have missed it, but they can now also let their voices out in a particular game…

Closing Words

This was it for December, as well as for 2018. We’ve made fantastic strides over the year, and it’s been a great pleasure for us writers to cover it throughout. We as a project and as a team are extremely thankful for every patron and user who has supported us and helped the project in any way, and we hope they’ll continue to do so in 2019.

We hope you liked this report and look forward to the next one! If you would like to contribute to the project, you can do so either by contributing code, helping the community or becoming a patron. RPCS3 has two full-time developers working on it who greatly benefit from the continued support of the many generous patrons. In exchange, patrons also get special support over on our Discord server and get access to early updates directly from our lead developers. If you are interested in supporting us, consider visiting our Patreon page at the link below and becoming a patron, or join our Discord server to learn about other ways of contribution.

2018, logging out.

This report was written by HerrHulaHoop, dio, Asinine, Megamouse, eladash and Digitaldude555.