Background

Shader compilation stutter is nothing new to most emulator users, especially on RPCS3. However it is worth clearing up some misconceptions that go around regarding how RPCS3 shaders work. I’ll try to quickly go over the history of shader compilation on RPCS3 and hopefully explain why the shader compilation stutter appeared and why some people believe RPCS3 did not have shaders before.

Shader complexity and custom vertex fetch

In early 2017, I embarked on a task to remove the very expensive vertex preprocessing step from the CPU side of RPCS3. This basically meant implementing all those custom vertex types and vertex reading techniques to the vertex shader and providing only raw memory view that the ps3 hardware would be viewing. This greatly improved RPCS3 performance, more than tenfold in some applications. This change is what made RPCS3 usable for playing real commercial games with playable framerates without needing HEDT system. However, the new fetch technique increased the size of the vertex shader and added a complex function to extract vertex data from the memory block. This made the graphics drivers take very long to link the programs, even without optimizations, likely due to use of vector indexing, switch blocks and loops with dynamic exits. Extra operations including bitshifts and masking were also needed to decode the vertex layout block. The code runs very fast, but the linking step is very slow. A shader cache system already existed before and if you ran an area for the first time, there was slight microstuttering that some users did not notice; its this stutter that got much worse. The solution to this: preload the shaders so that you don’t need to compile them next time. This lead to the infamous “Loading Pipeline Object…” screen and the “Compiling shaders…” notification.

Challenges

There are several challenges to tackling RSX shaders. First, the RSX is not a unified architecture like most programmers are used to today. It has separate vertex and fragment pipelines, both with their own separate ISA. They are also very limited and larger programs or more complex programs can result in very messy binaries. One of the largest problems is that the bytecode itself does not contain all the information required to run the program, extra configuration is configured via registers as draw calls are passed in. A good example is that the TEX instruction does not differentiate between texture types, but a texture configuration register exists that allows using the same program to read 1D, 2D, 3D, or CUBE textures as well as their shadow comparison variants. This means you can only know the generated shader once the texture register has been set up. There are other examples of things like these that make it so that you need the game to set up the program environment before the program itself is compilable.

Origins

Anyone who has ever looked at the RPCS3 program generation pipeline might have noticed that program decompilers are separated into program class and decompiler thread class. This is because technically asynchronous decompilation was in RPCS3 from the beginning but it likely never worked correctly and ended up being effectively synchronous (see this archive for the original RPCS3 code from 2011 to 2013). The goal was to rework how the programs are decoded and recompiled asynchronously. Before any of this would be possible, the decoder would have to be very efficient to lower cpu load, otherwise infinitely decompiling shaders would cause permanent flickering and graphics glitches. This is why I put in more effort into cleaning up the program decompilers beforehand to minimize unique generated shaders by reusing as much emitted code as possible.

Implementation

Once the vertex program analyser was improved to a point where most titles could be run with under 1k generated shaders, it was time to finally decouple the decompilation, recompilation and linking steps from the main renderer thread. This was pretty straightforward to set up for Vulkan which is inherently built with multithreading in mind, and a minor hassle on OpenGL where multithreading is more difficult to set up due to context spaces. It was expected that severe graphics glitching would persist until all programs were prepared but even with a bare bones implementation I realized that most of the needed shaders would actually be properly compiled after just a few seconds. This made the implementation usable in its incomplete stage, even without assistance of approximation shaders or interpreters. This was very good news and allowed this feature to be enabled by default as most users prefer the missing graphics for a second or two instead of completely freezing the emulator for a minute or so before all the shaders are compiled. This is particularly effective since not all shaders compiled even affect items that are visible to the user.

Results

The results are surprising even to me. The speed at which graphical glitches resolve is very impressive on capable machines. You can see a quick comparisons below:

Loading times and stutter are both reduced significantly when no pre-compiled cache is present.

Conclusion

While the compiler greatly improves quality of life when using the emulator, it is far from perfect. Lower end CPUs might lose some performance when the decompiler is active and even hitch and stutter slightly from time to time. Not as bad as before but it’s still a problem. There is also the issue of the missing graphics. This could be improved with approximation interpreters to loosely provide a placeholder program to assist in rendering.
While much work remains to be done, hopefully these improvements will help many users and fans in running applications on the emulator.

Regards,
kd-11,
Lead Graphics developer for RPCS3

If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. You can support lead developers Nekotekina and kd-11 here on Patreon. Your support will help speed up the development of the emulator so that one day every game will be perfectly playable from start to finish.

Also, come check out our YouTube channel and Discord to stay up-to-date with any big news.

The video above showcases various popular titles with a side-by-side comparison from 720p (native PS3 resolution) to 4k rendering with 16x anisotropic filtering in RPCS3. The difference is quite incredible in titles that have high quality assets. There is a lot of detail that just wasn’t visible at 720p. This feature is available for every PlayStation 3 game running on RPCS3. The only exception being that it does not yet work with Strict Rendering Mode. Games that require this setting will have to wait a bit longer before they can benefit from high resolution rendering.

Performance

Rendering a modern PC game in high resolutions such as 4k, while beautiful, is quite taxing on your hardware and there is often a massive hit in performance. However, since most of the workload for RPCS3 is on the CPU and GPU usage is low, there is a lot of untapped performance just waiting to be used. All processing is done CPU side, and as far as the GPU is concerned it is simply rendering 2006 era graphics (yes, the PS3 is 11 years old now). We’re happy to report that anyone with a dedicated graphics card that has Vulkan support can expect identical performance at 4k.

Anisotropic Filtering (AF)

High resolution support wasn’t the only thing that was added in this update! Another reason for such a massive upgrade in visual fidelity is having full 16x AF support. This greatly improves how textures can look, especially when viewed at an angle. Take a look at the below screenshots of Ni no Kuni as an example of the default AF vs forced 16x. The difference is especially noticeable on the ground inside the gate.

New Settings

After downloading the latest build of RPCS3, there will be some new options in the graphics tab:

Anisotropic Filtering

Anisotropic filtering can either be left on automatic or set to a specific forced value. Automatic mode means that it is up to the game to decide which level will be used, if it will even be used at all. It is recommended to simply set this to 16x as that greatly improves visual quality with virtually no loss of performance at all on any dedicated GPU.

Resolution

This is basically what resolution the game will think it is running at. For the absolute vast majority of PS3 games this is 1280×720. Even games that claim to support 1920×1080 usually do some kind of upscaling and don’t truly render at this resolution. Leave this at 1280×720 and let RPCS3 handle the high resolution rendering with the next option.

Resolution Scale

This determines how many percent above (or below) the original resolution RPCS3 should render at. 100% means that the native resolution of the game will be used. For example, a value of 200% means that for a 1280×720 game RPCS3 will render at twice the resolution: 2560×1440. 300% is 3840×2160 “4k” and so on. Even on a low resolution screen it can make sense to use a very high rendering resolution as the subsequent downscaling acts as a form of high quality anti-aliasing.

It is also possible to render at a resolution below that of the original game’s. A value of 50% on a 1280×720 game will render it at 640×360. While this will look terrible, it may prove useful for slow integrated graphics in some cases.

Resolution Scale Threshold

In general, do not touch this setting. This will not upscale framebuffers (bitmaps) that are smaller than the selected size which can fix missing graphics in rare cases when write color buffers are used. This is only a temporary workaround.

The Gallery

Below are high quality PNG screenshots of various games running at 3840×2160 or 2560×1440. We also took a few bonus screenshots of Demon’s Souls running at 7680×4320, though do note that these images are roughly 30 MB each.

Closing Words

High resolution rendering is only the tip of the iceberg when it comes to improving graphics quality. In the future, we will see custom anti-aliasing, texture scaling algorithms, and maybe even game specific patches by the community. Of course, work also continues with improving the emulator accuracy and general performance to enable more games to run, and to run well. If you like in-depth technical reports, early access to information, or you simply want to contribute, consider becoming a patron! All donations are greatly appreciated. As of last month, RPCS3 now has two full-time coders that greatly benefit from the continued support of over 600 generous patrons.Also, come check out our YouTube channel and Discord to stay up-to-date with any big news. This report was written by ssshadow, Asinine, and Talkashie.

As many are already aware, a lot has been going on lately with the new changes to the RSX (the PS3 GPU) emulation, dubbed vertex rewrite. This change moves a lot of vertex processing duties from the CPU to the GPU where they rightly belong and as a result there are massive performance gains especially with OpenGL but also with Vulkan in geometry heavy scenes.

Background

Most if not all users are probably aware by now, but dedicated graphics cards exist on a physically separate board. This means data has to be moved to and from it through the PCI-E bus which is quite fast. However, while it is high bandwidth, it is also high-latency. That means you cannot just send something over there and expect to get it immediately available for the next draw call. Instead, the GPU has to wait for data to be prepared and then signaled that data is ready for processing before drawing begins. This is a general simplification, but it helps illustrate the point. The RSX on the PS3 doesn’t work the same way however. It has near direct access to the XDR main memory on a PS3 and ‘pulls’ data directly from main memory as though it were local memory. It is somewhat similar to integrated graphics memory in this case. That means data is not ‘pre-packaged’ for transport to the PS3 GPU since the memory is virtually unified from the point of view of the RSX. When using Vulkan, drawing is not scheduled until the whole command queue is flushed mitigating the impact of transfer since data will likely have been uploaded beforehand, but for OpenGL this was a big bottleneck.

The second issue was that the emulator was doing a lot of computation on the CPU on how to read vertex data from main memory, essentially pre-packaging the data into formats easy for GPUs to use. This is a very slow process and also very memory intensive (hence the ‘Working buffer not enough’ crashes). Enabling a debug overlay with the old method shows some games taking up to 200ms to prepare vertex data for one frame (Hellboy: The Science of Evil). This is obviously not optimal. The impact could be lowered by using more threads for vertex processing, but with the number of threads already needed to emulate the PS3’s multi-core processor, it was a problem. Spawning 8+ vertex processing threads reduced the time spent processing vertices, but cost other threads to starve and performance would drop significantly. The solution was to shift the work to the GPU instead and not touch it in any way. Just copy the data block and the GPU could fetch the data it needed for itself, mimicking the behaviour of the real hardware.

Initial tests

The first task was to put this theory to test. I started by writing routines to identify continuous memory used by games to store vertex data and found most use interleaving to help speed up data transfer even on a real ps3. This is good since copying is very easy, boiling down to a single memcpy operation. I quickly got OpenGL to use this method whenever large blocks of data were in use and fired up a test case I had been using for some time to benchmark vertex processing – TNT racers title screen. The scene is very simple but throws up vertices using immediate draws and does over 2000 draws. This was a problem on rpcs3 where OpenGL was stuck at 10fps for a long time. Using the simplified processing, FPS went up to around 17, so the experiment was a success. Vertex upload times went down significantly from about 50ms per frame to under 10ms. However, the change in frame rate was not so impressive, so I had to do some more investigation.

Research and second attempt

After getting feedback from ssshadow that the performance boost was not as expected, I sought to find out how drivers handle memory writes to external PCI-E devices. For this I turned to the mesa open source drivers that I use on my PC and poured through the code. I confirmed the use of write-combined memory to improve throughput and went to work redesigning the rendering pipeline to make effective use of this. The simplified single writes to a small contiguous block already gave a good boost, so I tried reordering where the vertex upload happened to give it some time before the draw call requiring the data was issued. Memory writes were placed very early in the submission chain and other computations moved after this point but before the draw call. This way, the GPU could get the data before it is needed, improving efficiency of the renderer. With this change, I hit a GPU limit at ~25 fps which was 2.5x the initial test result and 10fps over the initial implementation. A driver update later brought the frame rate up to ~28fps but that is where my hardware maxed out.

Vulkan and disappointing performance

After the research was completed, I had a good idea of what to do to maximize throughput on Vulkan as well. I quickly applied the changes to Vulkan and refactored shared code between renderers to make future work easier. I quickly ran the same benchmark on Vulkan expecting 30fps but only got +1 fps gain; from 14 fps to 15fps. After measuring time spent in different parts of the pipeline I found another bottleneck that had been holding back Vulkan for the longest time – Every frame was waiting for the previous frame to complete before beginning rendering. This is not very efficient use of the graphics hardware or the API. Since the baseline framebuffer uses a doublebuffer setup, there are two surfaces to write to. There is no need to wait on previous commands if they write to a different surface. A quick restructure later and the renderer was rewritten to support asynchronous frame processing.

Final results

Combining the two optimizations and running the same benchmark yielded about 24 fps. This was about 10fps higher than the baseline performance provided on master. I fired up another benchmark – Hellboy: The Science of Evil and confirmed fps jumped from 3-5 fps before to 20-30 fps on both OpenGL and Vulkan. As an aside, the memory access behaviour of intel integrated graphics give 40 fps in that TNT racers scene using the new OpenGL backend – almost double the Vulkan performance of a 270x. This shows that things can be better with more optimization.

Before screenshots from Hellboy: The Science of Evil:

After screenshots from Hellboy: The Science of Evil:

Before screenshot from TNT Racers with Vulkan:

After screenshots from TNT Racers. The first two from a dedicated GPU with Vulkan and OpenGL. The last from integrated Intel graphics.

Another side effect of unifying vertex processing and handling things correctly was that many games that did not have any visual output before started working almosy immediately. Most notable cases here are Metal Gear Solid 2 and Metal Gear Solid 3. Shadow of the Colossus also got partially fixed as well as improvements to other games such as Sleeping Dogs and the Yakuza games.

Minor other bugfixes were also added that fixed Ni No Kuni’s flicking graphics when using Vulkan as well as broken depth in that title. Unreal engine 3 games also had broken color in intro cinematics and logos that were fixed as well.

Things were looking good but there was one more problem – shader compilation stutter was making things unbearable.

Shader cache reorgnization

Rpcs3 has always had a JIT shader cache that slowly builds as the emulator runs. It was however very broken on Vulkan causing programs to constantly rebuild. This manifested as microstutter before where the game did not feel smooth although the fps counter showed high fps. With the new more complex shaders, the stutter went from a few milliseconds per shader to several seconds in some cases. Even after fixing the microstutter on Vulkan due to bad key hashing, it still took unbearably long to generate the internal cache into something useful for gameplay, interspersed with minute-long pauses in places. It was quickly apparent that something had to be done. As such I decided to have the shader cache dumped to disk and preloaded when the emulator was started up again. This made significant changes to the feel of games run from within the emulator. A few workarounds were needed to make intel work at all. Its still not working 100% but improvements will be coming.

Bugs and technical issues

While the changeset does improve the core rendering pipeline a lot with the new systems, it brings with it new baggage. First is the aforementioned increase in shader compilation times. The second more serious issue is more relevant to nvidia users – high memory usage when the number of precompiled shaders rises. This issue was brought to my attention by a user on discord who mentioned that Cemu works the same way and that nvidia users experience the same problem with memory usage going up very high. On rpcs3, this means you can see the emulator consume 5+ GB of RAM when compiling the shader cache. An interim solution for those with lower RAM would be to clean the shader cache periodically until we find a suitable workaround. More information can be found here.

Lastly, intel drivers (at least on windows) have buggy glsl generation. I have a workaround in place specific to intel but it wont fix all problems on that platform. More work is certainly needed.

What does this mean?

First, it means OpenGL is now within striking distance of Vulkan in terms of performance and faster in some cases. Vulkan however suffers more from the first-time shader compilation stutter. This will be improved in the near future, but for now, you may want to give OpenGL a try if the stutter is too distracting or avoid the new nightly builds for now. Overall Vulkan is still faster now with the reworked framework. I have seen suggestions about disabling the shader cache – that wont help with the stutter since its caused by the linking step that is done by the drivers. I would ask those negatively affected to be a little more patient as we work to resolve the issues.

Closing Words

If you want to help out and make RPCS3 progress even faster you can check out the Patreon page here. Right now we are coming closer and closer to the $3000 goal of having me, kd-11, work on RPCS3 full time which would greatly increase the rate of progress in the future.

What Was the Issue?

There were a lot of problems. Back in January, quite literally nothing was working. RPCS3 would crash instantly upon booting any game, if the program itself would even start at all. Moreover, additional functionality like the debugger, framerate counter, and firmware installer were completely broken too. While many of these auxiliary issues were quickly identified and fixed, the fact of the matter was that almost every game would hang after running for a few seconds. This turned out to be much more difficult to fix. This was caused by several different bugs in thread synchronization which were fixed continuously in the past few months. Finally, one last relatively small commit in early April fixed the last bug and suddenly RPCS3 on Linux went from basically nothing to Demon’s Souls and [redacted]. Or so we though, but we quickly found out that the LLVM recompiler was completely broken for a lot of users who just got completely nonsense errors. We encountered strange and esoteric bugs and oddities about LLVM and how various Linux software, including the Mesa drivers, were using it. These problems made RPCS3 unusable for a lot of people. A lot of false flags and red herrings later the bugs were fixed not by changing any code, but by using rare compiler flags. RPCS3 on Linux is now working as intended for everyone, including AMD and Intel graphics users with modern Mesa. Even Vulkan with Mesa is now working!

PlayStation 3 Games on Linux

Below are some popular PlayStation 3 games showcased running on Linux. Performance is about the same as on Windows, perhaps even a few percent better in some very intensive games like [redacted]. But take note: These images were either captured on a laptop with a very old i7-2670qm CPU, or a fast desktop with a i7-4770 CPU.

Demon’s Souls

Demon’s Souls on an i7-4770

[redacted]

[redacted] on an i7-4770.

Catherine

Catherine on an i7-2670qm.

Other Games

1942: Joint Strike (i7-4770), Ni no Kuni (i7-4770), Hatsune Miku: Project DIVA F (i7-2670qm), and The Elder Scrolls IV: Oblivion (i7-2670qm)

Download and Instructions

Builds will be delivered on rpcs3.net in the form of AppImages. With this packaging method every dependency is already taken care of and no special prerequisites are needed. Moreover Travis-CI will build every single new merged pull request, package, and upload.

Be sure to read the quickstart guide as you required to install the official PlayStation 3 firmware in RPCS3. The guide also covers how to run games in practice. It has been updated to highlight one important configuration difference compared to the Windows version. Essentially log files, game data, save files, LLVM cache, and everything else like that will be located in ~/.config/rpcs3/, that is the hidden folder .config in your home directory.

The Future of RPCS3 on Linux

While compatibility and performance are now on par with Windows builds, there are still a few features missing and those are listed below.

Controller Support
Update: Controller support with evdev was merged recently. Refer to the quickstart guide and how to configure evdev for more information.
Input on Windows is supported via the keyboard, native DualShock 4, XInput, and mmjoystick handlers. The two latter are Windows specific input methods meaning that on Linux you can’t directly use any kind of gamepad with RPCS3 except the DualShock 4. You could map keyboard keys to your controller as a workaround but this isn’t optimal. Support for evdev input, that includes for example DualShock 3 and the Xbox 360 controller among many others, is being worked on right now in this PR.

Wayland
RPCS3 works fine with XWayland but perhaps this isn’t optimal for everyone. However RPCS3 has moved over to Qt, and a lot of the Wayland groundwork is already in place.

Closing Words

RPCS3 is free and open-source software. Lead developers Nekotekina and kd-11 are working on it full time thanks to the generous funding from nearly 500 patrons. Moreover progress is greatly accelerated thanks to the help of everyone else that has contributed to RPCS3.

There are over 2000 games listed in the compatibility database, and only about 20 of those games were tested on Linux by the developers. While a selection of advanced games like Demon’s Souls and [redacted] seem to run fine, it can not be ruled out that there may be issues with other games that only happen on Linux. That is why your help is needed, please test your favorite games with different hardware and software configurations and report your findings on our Discord server or forums.

You can support Lead Core Developer Nekotekina and Lead Graphics Developer kd-11 on Patreon with the link below.

First up is, obviously, the actual transition from the user interface toolkit WxWidgets to Qt. This transition has been a long and time consuming process, but it added a host of new functionality, both visible and not so visible. A non-exhaustive list of the transition pull requests, the main of which can be found here is:

• – Appveyor and Travis now build with Qt project, thanks to hcorion. This switches the nightly builds, and eventually the linux builds, to use the Qt interface.
• – Made some design changes to the GUI, such as progress bars and taskbars now showing percent completion and slight improvements to the Vulkan/DX12 adapter selection box to make it easier to use.
• – Added support for layouts, which allows you to move your docked widgets as you please.
• – Added support for booting games in fullscreen.
• – Added a Welcome screen to point new users to the Quickstart setup guide.
• – Support for themes! An example can be found further down.

As with all major changes, this caused a few hiccups, but they are being worked on as they are found, and issues can be reported here.

This transition to Qt brought a wave of GUI improvements with it. First up was a recent games list, implemented in #2843 and #2847. This saves the last nine games launched in an easy to use list that has hotkeys, so that those with large libraries, or those using Blu-ray disc games, can easily launch their favorites. The list can be frozen, though items cannot yet be pinned.

Second was a new viewing mode and a tool bar. The game list can now be viewed as a simple grid:

As you can see in the screenshot provided by user Talkashie(which also shows a custom made theme!), the new toolbar allows for searching of large libraries, as well as directly filtering the list by game types. An example being that you can filter out audio/video apps and home apps, or you can choose to view only HDD games. Thanks to user rutantan for creating the very nice button icons in the toolbar.

Yet another newly added feature was the VFS dialog, which was added in #2863. This allows users to change the directories used for things such as dev_hdd0, which enables sharing a single directory throughout all rpcs3 installations. This is a very useful feature for quickly testing games between commits going forward, as it allows users to boot games from the game list quickly, rather than having to either create a second install of the game or manually boot it. This feature shouldn’t be confused with adding Blu-Ray disc games to the game list, which will come at a later point in time.

#2876 revamps the debugger and CgDisAsm GUIs to both look better and be more functional. This includes improving resizing of the debugger, replacing tabs with a side-by-side view in the CgDisAsm and adding drag and drop support for fpo and vpo files. More details can be found in the actual PR. Improvements to the debugger are ongoing.

#2890 redoes the gamelist categories to be more faithful to the original PS3. It allows all bootable categories to boot, and the filters should work as intended now. Several miscellaneous categories have been condensed into an “other” category for the time being. The icons in the toolbar have been updated as needed as well.

One of the most exciting commits this month is #2924, which connects the stubbed save manager to the actual save function. Simply put, this allows for the creation and use of multiple saves. This is only an initial implementation, but it enables one of the biggest missing features of a real console, and is a huge step forward for the emulator.

Several other pull requests not mentioned here fixed bugs, restored forgotten functionality, and improved the new features mentioned above. Several other GUI features are in the works, including properly featuring Blu-Ray disc games in game lists by creating a directory for them. In addition to existing features being improved and new features being implemented, Megamouse is working with lead web and graphics designer DAGINATSUKO to rework the GUI into a much smoother and user friendly experience. A preview of some of the upcoming changes can be seen below. Bugs will be fixed as they are found as quickly as possible, and hopefully new features will keep on coming.

A look at the rework of the main window to the right compared to the old main window to the left.

New Keyboard controls setup screen. Much more intuitive than the old one.

Settings GUI updated to look better, and includes tooltips for what options actually do.

Closing Words

This report only scratches the surface of the work that was done to bring RPCS3 over to Qt. flash-fire, Megamouse, hcorion and others worked hard over many weeks to make RPCS3 better in the user interface department. They will also continue to do so in the future as there is still more work to do.

You can support further RPCS3 development on Patreon here.

This report was written by twdarkeh and edited by ssshadow.