Ok, after extensive testing, I can confirm that I no longer remember anything about how Windows works, lol. But, I have learned a lot.
I believe that if someone can work out the correct incantations on the Window command line (using Powershell 4, not Powershell 5+, because Powershell 4 is what the windows system github actions uses is running) to do the following, I can get it to build something:
1) Download this zip file:
https://github.com/sbcl/sbcl/suites/14610349674/artifacts/828195592 (preferably into the parent of the pwd).
2) Extract the contents (anywhere other than the pwd while still on the path will do, alternatively you can add wherever you extracted it to the path).
3) Run the installer (.msi file) contained inside that zip file.
4) Download this file:
https://beta.quicklisp.org/quicklisp.lisp (preferably into the parent of the pwd).
If someone can work that much out, I feel confident I can get from that step to a working bin file via Github Actions, at which point I'll zip that up and post it here for testing. Assuming it works, I'll have it auto-build a Windows zip file with every release.