I am on a new computer again and was going to go through the install process for tidal, and I became curious again about the possibility of having a simple static binary for tidal cycles.
I was poking around the source code and found main/Main.hs, which appears to set up a haskell interpereter set up to run tidal cycles. However, it doesn't appear to have an entry in tidal.cabal or have its own .cabal file. I tried adding a section for it and running cabal build:
executable tidal
main-is: main/Main.hs
Build-depends:
base >=4.8 && <5
, tidal <=1.10.1
, hint >0.9
, exceptions < 0.11
The values there are sort-a guesses as it has been years since I've messed with cabal.
This successfully compiled and built an executable tidal that I could run, but I wasn't able to get code to work:
[TidalCycles version 1.10.1]
Installed in /home/ludens/.cabal/share/x86_64-linux-ghc-9.6.6/tidal-1.10.1
Listening for external controls on 127.0.0.1:6010
[tidal] starting...
[tidal] modules loaded...
[tidal] ready
Waiting for SuperDirt (v.1.7.2 or higher)..
Connected to SuperDirt.
sound "bd"
type : ControlPattern
value: (pure s: "bd")
d1 $ sound "bd"
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-link-1.2.0/build/libHStidal-link-1.2.0-inplace.a: unknown symbol `_ZGVZN16link_asio_1_28_015system_categoryEvE8instance'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-link-1.2.0/build/libHStidal-link-1.2.0-inplace.a: unknown symbol `abl_link_time_at_beat'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzmlinkzm1zi2zi0zminplace_SoundziTidalziClock_SetNudge_con_info'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzm1zi10zi1zminplace_SoundziTidalziStreamziUI_streamList1_closure'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzm1zi10zi1zminplace_SoundziTidalziSafeziContext_streamReplace1_closure'
tidal: ^^ Could not load 'tidalzm1zi10zi1zminplace_SoundziTidalziSafeziBoot_d1_closure', dependency unresolved. See top entry above.
GhcException
GHC.ByteCode.Linker.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
tidalzm1zi10zi1zminplace_SoundziTidalziSafeziBoot_d1_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session. Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please report this as a GHC bug:
https://www.haskell.org/ghc/reportabug
I get the feeling this could be fixed with some changes to the executable section, or perhaps main/Main.hs just needs some attention, but this makes me wonder.
What's the story behind the main/Main.hs file? Would this make it feasible to make a tidal binary and put it on the release page, maybe even package it?
Edit: I was on the latest commit of main, d5fb53eca2c699c11d8d31a6a6aa50da6bba5aa7. Going to try a tagged release.
hey, so i’ve spent 3 months in 2022 to try and package tidal into a static binary as part of the google summer of code. i had it working with a version of tidal that didn’t have ableton link support yet (but it’s still very tricky since hint needs to be basically “packaged manually” by copying a lot of the files from the cabal store to a location the executable knows about).
but since tidal-link i wasn’t able to fix this approach.. you can read through this issue here:
it would be amazing to get this to work but i fear it might be more work than you expected.. i’m happy to answer any questions but can’t commit to working on this much unfortunately
Wild idea: since we can get ghc/cabal to produce a .o file, I wonder if it makes sense to jump directly to using the GHC c api via C++ or another language and circumvent some of ghc's linking weirdness. I will do some research and see if it is possible to statically link ghc via C bindings, and what we would actually have to do in order to get it to find haskell symbols.
It's been a while since I touched this code. I will have a look again.
Still, I don't think one would magically get a fully self-contained executable here: hint will do some kind of compilation, so it needs to read libraries (.hi files, .o files) from disk, these need to be installed in proper places (so ghc-pkg or equivalent finds them).
The linking issues appear to be related specifically to tidal functions. Ideally, these would be statically available to the ghc environment, so if a user only uses tidal functions in a particular .hs file, they wouldn't need any extra infrastructure. If they used external libraries, all bets are off.
Getting your Haskell executable statically linked without Nix · Hasufell's blog
A guide to statically linking a haskell executable, using docker and alpine
linux. The trick is the need to statically link with musl libc, which
apline and void linux both use. However, this also means either finding a
way to build all dependencies from source, or relying upon the alpine package
ecosystem to have the necessary packages. Thankfull, I think the ones required
by tidal cycles are either built from source already, or are more-or-less the
default ones that haskell requires, so likely to have them available.
$ cabal --version
cabal-install version 3.14.2.0
compiled using version 3.14.2.0 of the Cabal library
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 9.12.2
$ uname -morps
Linux 6.17.6-300.fc43.x86_64 x86_64 unknown GNU/Linux
To be clear, if you run the executable and evaluate d1 $ sound "bd", you don't get these runtime issues?
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-link-1.2.0/build/libHStidal-link-1.2.0-inplace.a: unknown symbol `_ZGVZN16link_asio_1_28_015system_categoryEvE8instance'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-link-1.2.0/build/libHStidal-link-1.2.0-inplace.a: unknown symbol `abl_link_time_at_beat'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzmlinkzm1zi2zi0zminplace_SoundziTidalziClock_SetNudge_con_info'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzm1zi10zi1zminplace_SoundziTidalziStreamziUI_streamList1_closure'
tidal: /home/ludens/Code/tidal/dist-newstyle/build/x86_64-linux/ghc-9.6.6/tidal-1.10.1/build/libHStidal-1.10.1-inplace.a: unknown symbol `tidalzm1zi10zi1zminplace_SoundziTidalziSafeziContext_streamReplace1_closure'
tidal: ^^ Could not load 'tidalzm1zi10zi1zminplace_SoundziTidalziSafeziBoot_d1_closure', dependency unresolved. See top entry above.
What happens if you run ld against the resulting binary?
If the issue in hint was diagnosed correctly, and if I understood it correctly, the issue without the -dynamic flag has to do with how the ghc linker finds certain C++ symbols, so it makes sense it works when using the system linker, as this is what ghci does according to that thread.
I think ghc/cabal have a flag to control linking to particular libraries: -optl-l<lib>. Would it be possible to specify only using the link library dynamically? -optl-llink or something?
My guess with the segfault would be that the binary needs a few shared objects which do not exist on your path: (note that I got rid of the whole crazy hash and compiler version in the names)
libHStidal-1.10.1.so
libHStidal-link-1.2.0.so
libHStidal-core-1.10.1.so
But we could solve this issue with a .deb/.rpm package, so it would at least be much easier to get up-and-running. Having only libHStidal-link-1.2.0.so (or ableton link) be dynamic would help reduce the number of files needed as well.
probably due to version mis-matches (version of source from which I built tidal-main vs. version that I had installed globally). went away after cabal install --lib
.. report upstream
maybe. I have no idea. I built this Main.hs in order to replace IO with a safe shim so that (a ghci session runnig) tidal could be exposed for remote collaborative editing. That worked, but it was dynamically linked. Static linking was not the goal.
Ah I see, I realize now that the main.hs file in tidal is maybe different from what gets referenced in the hint thread. Will try some more stuff this afternoon!
Cross-posting here; someone made ghci run in wasm, and I think it’s possible to load modules somehow. If we could distribute tidal this way, we could distribute a wasm blob with a vscode extension for 1-click install.
I got some help from the ghc-dev mailing list, they recommended creating a package-db for tidal's packages; I think this may have been tried already when experimenting with building a hint application and deploying it to another platform.
I also thought that maybe, if I could find the right function which actually loads a haskell module, I could use template haskell to evaluate it at compile time, and thus have the module inside the application, but did not get very far as the ghc/hint libraries are a bit beyond me.