Updating the Tidal website and documentation

The main tidal website is a wiki, using the same 'mediawiki' system as wikipedia. There's info about how to contribute here.

The old website is here, and looks a lot nicer. The documentation there is out of date now but has automatically built examples that haven't yet made it to the new website.

I thought I'd start this thread to go through what needs to be done to make the wiki friendlier.. Some points for potential discussion:

  • Web design / graphic design - customising a mediawiki theme?
  • Information design - how could we better organise the wiki? Do we have the 'four kinds of documentation' covered?
  • Translation - some parts are translated, could we be more organised about that?
  • Are there better alternatives to mediawiki? (I'm a bit resistant to moving it all again but maybe there's a new system we could look at - it'd need good internationalisation/translation support though)
3 Likes

It's been I while since the last time I visited the website, giving it a look now I remember what confused me a lot when I used it frequently:

  • The home page is tidalcycles.org, but the title reports "tidalcycles userbase", but the userbase is another page. What
  • The homepage structure is not so intuitive, it starts with a youtube video preview without a context, then the introduction text and the rest of the page. The more important link (userbase) is not so visible... Maybe I would move the "getting started" list of the userbase page into the homepage including a link to the userbase itself
  • The userbase page has too many entries, also a duplicated element.

The graphic design could maybe improved but I think it's clean and clear enough (I'm not a design expert though).

I would gladly help with the re-evaluation, I would like to know what tidal people think about that

3 Likes

Another little thing, but it hurts me a little :smiley:
Can the index.php part be removed from the url? It works also without it, but it appears automatically on the welcome page and after every link followed

I fixed that one!

Good thoughts on the other stuff. Maybe we reorganise things a bit, make the front page simpler. Lets make a sketch in the 'New' namespace:

Merry Xmas all!

My thoughts on (technical) documentation (so, not on the web site in general)

  • when reading documentation: I like it when documentation and code are close (i.e., automatically refer/link to each other).
  • when writing documentation: I like it when there's just one source of truth (so, no manual copying of texts) - and the Haddock section for each type or function seems a good place.
  • documentation can contain executable examples and test cases https://hackage.haskell.org/package/doctest - then the machine checks that documentation is correct. Like magic!

But this implies that each change in documentation - is a change in the code, so the doc author needs access.

If there is anything in the direction of automation, e.g., cross-referecing between Wiki docs (or their index) and Haddock, I can try to help. But that's perhaps better discussed in "innards", or in the issue tracker?

2 Likes

I made some experiments... shrunk the heading text from 1 to 2, moved the youtube thumbnail at the end of the first block (in that position looks way better from a mobile device), reported the principal "userbase" voice into the "documentation" block (renamed in "getting started").
Maybe the "Upgrade" and "Start tidalcycles and superdirt for the first time" could be inserted into the "Installation" page and "Installation" could be renamed in "Install / Upgrade".

Referring to the "four kinds of documentation" you linked, I like when, in a software home page I find links that address me to:

  • how to install and execute it
  • a tutorial (first steps, simple esample)
  • full documentation (or reference)

PS: i noticed that the mediawiki edit mode is not so user friendly, no syntax highlighting, no live preview... it's updated to the last version?

1 Like

I totally agree with you, the best documentation is the one that goes hand in hand with code.
Haddock generated documentation from the code would be the best place for the complete function reference, leaving all the rest into the wiki.

Thanks for the updates!

There's a few reason why we didn't go with docs generated from code.

One is difficulty for lay people to contribute. However not many passersby have contributed to the wiki.

Another is internationalisation - the wiki allows translations of pages. Is that possible with hackage style docs?

Then there is freedom of organisation.. I think documentation probably needs a different structure to the way the code is organised.

Could we work around those last two issues?

I've been using markdown for documentation for years now and haven't looked back. These can live in a separate repository that anyone can commit to, and the markdown can be rendered to power a custom docs site.. maybe a docs.tidalcycles subdomain?

Here is an off-the-shelf custom UI powered by a directory structure of markdown files: https://docsify.js.org/

In theory, it would be possible to turn any tidal code example into a working example using the web version of tidal?

Edit: it does allow for translations, in the way that you can create sub-directories split by language: https://docsify.js.org/#/cover?id=multiple-covers

I've updated the wiki to the latest version. I think the nice editor is an extra plugin that we're missing though, I'll try to look into it.

I noticed you remove the translation tags from the New: version of the front page.. We need those I think otherwise the translations will be lost.

Uhm...sorry, I didn't know that tags were useful for translation, it's the first wiki with translation that I modify :slightly_smiling_face:

Will fix it later

Thanks. I think they mark blocks of text so if you reorganise things, as long as the same tag is above the same text the translations should still work. If the text itself is edited then I think everyone sees the original (probably English) text until new translations are done.

I've spent quite some time on the french translation a year ago. It's far from perfect but at least it's something to start from. I think that nobody uses it though. I'm not sure about that but it seems that the current behavior of the wiki is to always direct the user to the english webpage by default, no matter what. The other translations are always a click away. If you can read both english and [target language], reading the english doc will most likely be the faster route from opening the browser to getting the right info.

I've stopped contributing when it became clear that translating more pages to french would break the structure of some pages that are browsed very often by the community at large (e.g. the functions list one). There is also the fact that translating without getting feedback from other peers is difficult.

Here are my suggestions for renewing the effort concerning documentation translation:

  • Writing documentation about how to contribute / how to help with translation. People might feel the need to discuss many topics that are essential to the translation task: how far can we depart from the original page? for what reason do we diverge from the source language? should we look at the english pages as a "definitive reference" or as a "general outline"?
  • Structural overhaul of the wiki. Category pages are great for listing contents because they are systematic. However, they list everything in multiple languages and lead quite quickly to information obfuscation. The wiki should also try to detect user's current language and serve the local page if possible.
  • General effort to gather all the cool things that have been discussed on the TOPLAP chat / Tidal Club forum in the wiki. It's only my opinion but information about Tidal is getting more and more scattered around the web as it gradually becomes a popular language for music improvisation.

I would be glad to participate to this effort and I might find some other live-coders that are willing to work on it along with me (at least for the french translation and general organisation).

PS: fixed a few typos and grammatical errors here and there.

2 Likes

I had a good go at getting the visual editor to work after upgrading everything but kept hitting strange errors.. I think our current default theme isn't compatible with it but even on the wikipedia-style monobook with other plugins disabled I couldn't get it working. Hm!

Thanks Raph for thoughts and all this past work. It's something I struggle to help with as I'm sadly monolingual. I did think that the wiki used the language sent by the web browser, it's very likely I'm wrong, or maybe it defaults to English for logged in users, or something like that?

Yes agreed that things have become scattered. There's a 'friends and relations' page on the wiki which has some links but is rather hidden. It'd be great to do a reorganisation and have translation as part of that - because editing pages that are already translated will likely result in some translations needing editing too.

One way to do it would be to setup a "template" diagram that describes what the Tidal wiki should look like (main categories, sub-categories, branching out on various topics). There are some topics that are absolutely non-avoidable: installation, troubleshoting, explaining Tidal's architecture (or as I call it, the Tidal/SuperDirt/SuperCollider lasagna).

As translators will stray further and further away from these topics, they could ideally regain some ability to organize content to their liking in the target language if it makes more sense to do so. This way, only one part of the content would have to follow closely the english reference and for good and obvious reasons (following the natural evolution of the language and the root technical documentation).

I like the four kinds of documentation way of seing things. Building a template about these four aspects could constitute a great grounding for future documentation efforts.

2 Likes

I think the transate tags are actually used in a strange way...
e.g. (copy-paste from the actual welcome page):

<!--T:6-->
Tidal is embedded in the Haskell language, although you don't have to learn Haskell to learn Tidal - most tidal coders have little or no experience in software engineering.

= Documentation = <!--T:7-->

<!--T:8-->
Enter the [[Userbase]] for installation info, reference material, tutorials and guides!

As it is now, the section 6 contains the Tidal is embedded... paragraph, plus the Documentation title, while the section 7 is just an empty line, or am I misinterpreting?

Back to the documentation discussion...

I like this mindset, but I think that the basic function reference should be written and mantained by developers, as it's generally done for public api documentations.
If the code structure does not fit the domain... then it's time to refactoring :slight_smile:

Regarding translations, I understand the need for those, but for example, I avoid every italian (my native language) translation when it's about IT stuff. Often it's really weird to read meaningful english terms translated into a lot of meaningless italian words.

1 Like

I'm wondering if we can have a best of both worlds situation..

Put reference documentation for each function into the source. Compile that into a folder with a flat structure, one file per function, e.g. jux goes in ref/jux.md no matter where it is in the source. The only exception is internal functions not intended for use outside the source, which could be somehow tagged and compiled into ref/internal/nameoffunction.md. An index to functions can then be generated, with a separate list of documented internal functions for the curious.

Then tutorials, explanations and how-to guides can be written directly in markdown. Function names in backticks can automatically linked to the reference material.

I was just admiring the patchbox os documentation: https://github.com/BlokasLabs/patchbox-os-docs/

It's markdown based, built using mkdocs.. Which looks similar to docsify.js that @mashaal mentioned although builds static files rather than rendering markdown on the fly? Not sure of the advantages/disadvantages of these - opinions welcome!

What I especially like about patchbox os docs is the use of (unlisted) discourse threads for comments:
https://blokas.io/patchbox-os/docs/first-run-options/

That would be amazing to include.

It would be a fairly huge effort to move wiki content over to in-source and markdown docs, especially including translations, but it would be some work to reorganise the wiki as-is.

One advantage of all this work is that we could build integrated editor docs, as well as a documentation website.

6 Likes

Using a discourse forum as a commenting system looks straightforward and well supported:

1 Like

MediaWiki is fine, but at the same time I think the Markdown approach (either with mkdocs or docsify) is superior in most aspects. I like the idea of documentation as plain text files stored in directories in a git repo, instead of a SQL database, it's easier to migrate to other technologies if needed, easier to backup and much more transparent.

I18n is possible if you store files in directories per locale, it's just a matter of convention (i.e having a theme that understands that structure). But I personally never threaded there, need to research a bit.

You can also edit any doc from the GitHub editor just by having a GitHub account. I have seen themes where each document page has a "Edit this page" link, which takes you to the Github edit URL for that specific page, so it's very direct. To simulate the same behaviour of "allow any edit" as in wiki, we could have a Github bot for auto merging those kinds of PRs too.

I can help with this if we decide going back to markdown again (I know it's a lot of work, but maybe some of that can be automated?).

But regardless, I agree with all your points mentioned (I really like that four kinds of documentation post).

It'd be nice to have a guide on how to contribute to the docs too, as @raph said (about adding new entries, updating examples, translating, etc). About translations, I'm not sure if there is a way to show or display all untranslated pages, or see how's the progress of a particular language, but I guess we could be more transparent of the whole process, to prompt more users to translate.

2 Likes