Audio is the most crucial part of Shr3D. It is essential to get a very low round trip latency (≤ 10ms). Here is a overview:
Note: SDL is used whenever I have not yet spend the time to implement the platform specific code.
Update: check out .shred, the Supreme Heavy Riffage and Effect Description file format specification.
A long term goal is to create a new file format as a replacement for .psarc files. As the PlayStation ARChive name suggest it is a archive for other files.
Obviously the main reason why Shr3D supports .psarc file is because there is a huge community creating them and there exists decent tooling for creating your own files.
Shr3D does cache metadata and album covers of .psarc files which helped with the biggest issue: Performance. But I think a new file format would be even better.
Here is what is so bad about the .psarc file format:
We need a simple archive format. The containing files should be human readable.
I think it would be good to really think hard about what to keep from .psarc and what should be added. I would like to nail this format this once and for all and not be another example of xkcd 927.
Right now I'm leaning towards .zip since it is so widly used. The best thing about it is that no external tools are needed for editing the content on Windows. The .zip extension would be renamed to .shred. Shr3D could play/import them by simply double clicking.
Human readable files would be nice in case you are not satisfied with your guitar tone on a file from someone else. You could have patch files that only overwrite parts inside the original .shred file. In case you want to share your improvements you could have the option to make your local overwrites permanent.
The .shred files would usually be one song per file with as many arrangements as you want. But you could also have a .shred file that is a collection of other .shred files. The naming schema would be similar to .psarc, 'artist-name_song-title_v1.shred'.
One preview audio file + at least one audio song file: The .ogg format would be the straightforward choise. Modern lossless formats do exist, but there is nothing that could be gained when converting the existing .psarc files (.ogg audio = lossy compression) into a new audio format. Multiple audio song files could be allowed to make it possible to have .ogg files with instruments like lead guitar or bass removed. Note: I dislike that many existing .psarc file have an additional period of silence at the start of the audio file. I would prefer to have use configurable silence in Shr3D that delays the audio of all songs equally start of all songs.
One Album cover file: There is no universal compressed image format that works on all GPUs. Thats why album cover must use the .png file format (you can't really go wrong with it). A resolution of 256x256 RGB should be a good compromise between file size and quality. .ogg files do have a metadata artwork section, but we shoud not use it.
One Lyrics file: Lyrics are simple. We have the time when it will be shown, the duration and the text itself. A file with lines like "123.456,0.068=Hello+" for each word will do. A '+' will indicate a line break. A '-' will skip the space after the word.
One song info file: I also choose a extra file over the .ogg metadata section. Simple "key=value" lines like "albumYear=2004" will do.
Arrengement files: These are complicated files containing the note data, phrases. The new files would be human readable of course. I want them to be diffable in case you want to do a fix on an existing file. The obvious goto format for this kind of data would be JSON. I've looked into it, but I decided to stick with .ini here too. Reasons:
The renderer in highway.cpp is quite simple.
I would call it an immediate mode renderer.
By that I mean all geometry is send to the GPU each frame.
The nice thing about it is that changing any settings like the view distance, note shape or string color does not cause any buffers to be reinitialized and it will not cause any performance hickups.
The tradeoff is that it is not very fast in generall since we do a draw call for each geomertry.
Thankfully our frames do not contain a lot of it and we can mostly get away with calling glDrawArrays about 300 times per frame at 1000 fps. It's crazy what you can get away with on modern hardware these days.
When using Shr3D on the Web there is quite some overhead involved and you are roughly only getting about 5% of the fps that you would get on the desktop.
This is a problem and highway2.cpp should improve it eventually by sorting the geometry in a frame based on the Z coordinate, using VBOs and instanced rendering.
Shr3D v0.4.2 got a record button!
I added it because I don't have a ASIO audio interface with loopback.
Without it you can't caputure the sound you hear on your headphones.
The new recorder solves this problem if you only want to capture audio.
You would still need loopback for live streaming Shr3D.
As a workaround I used two audio interfaces in the past.
The second interface was hooked up to capture the main output of the audio interface I used on Shr3D.
I would not recommend this setup because you will add some noise to you recording.
Here is how the recorder works:
After toggling the button, the audio signal you hear on your headphones is written into a buffer.
When the song ends, this buffer will be written to disk.
It will be put into this nice directory structure:
recorder/artist/year-album/song/YYYY-MM-DD_HH_MM.wav.
This is excellent for tracking your progress over time.
When I was added this features, I looked at Reaper.
As soon as you hit record a .wav file being written.
It uses 24bit integer as the format.
For Shr3D I also planned on using 24bit integer as well, but it is super easy to have a clipping signal without noticing it.
Shr3D uses 32bit floats internally.
It does not do much mixing at all
Audio signals are simply added together.
By default music, instrument and metronome all have a volume of 0.5.
The signal could peak at 1.5.
Even worse, the instrument volume is just the source signal.
There is no limit on the volume of the signal after the fx chain.
I have no idea how Reaper is dealing with this issue, but lowering the overall volume when the song ends does not make much sense since the signal peaks could be so unpredictable.
I scraped the idea of 24bit integer and just went with 32 bit float .wav files.
This is super nice because there is no code needed for doing any float to integer conversion or dealing with clipping the signal at 2^24.
And most importantly you can't ruin your recording anymore.
Most big corporate logos today are simple and boring.
My favorite open source project logo is the old LLVM dragon logo. I think it is just perfect. I think it is super recognizable because there are not many logos like it around. I want Shr3D to have a similar badass logo, but I'm lacking the drawing skills for it.
I generated the current logo when AI image generation got to the point of getting usable. I'm not sure how I should feel about AI art. It is awesome because I'm able to create about without much effort. On the other hand the web is already full of it.
Anyway, I think the current logo could be improved. I would like to have a uniform set of logos, banners, fonts, etc.
From my gear video, you might have seen my main guitar featuring a Boss GK-5. I wanted to have the same finger tracking on my bass guitar. This time I installed a Roland GK3 kit into it. I snapped some pictures of the process:
