Event Highlighting

Innards
1

I'm playing around with a Tidal editor and trying to get feedforward-style context highlights working. Right now I'm running a Tidal boot file that contains the following editor target:

let editorTarget =
      Target {oName = "editor",
              oAddress = "127.0.0.1",
              oPort = editorPort, -- From an ENV variable
              oLatency = 0.02,
              oSchedule = Pre BundleStamp,
              oWindow = Nothing,
              oHandshake = False,
              oBusPort = Nothing
             }
             
    editorShape =
      OSCContext "/editor/highlights"

Then, I'm passing it into my config with:

tidal <- startStream (defaultConfig {cVerbose = True, cFrameTimespan = 1/20}) 
                       [(superdirtTarget, [superdirtShape]),
                        (editorTarget, [editorShape])]

Sure enough, this produces a stream of OSC messages with the pattern ID and the location of a highlighted range. However, it's not obvious to me how you go about matching highlights with individual mininotation strings. Any tips? Is there something else I should be doing?

Like, if I have the pattern d1 $ s "sn bd" # (rotR 0.5 $ n "15 16") and get a message to highlight the range [1, 3], how do I know which symbol to highlight?

My guess is relying on the order that events with the same timestamp come in, but that doesn't seem reliable since UDP makes no promises about dropped packets, etc.

2

Hey @archaic.tech

These numbers will tell you the column/row of the position within the mininotation. There should be four numbers to give you the start/stop. I think the row is always 1, because mininotation doesn't normally go over more than one row.

This is not so useful by itself, as you don't know which mininotation string it's referring to, and therefore whm00ere in the pattern should be highlighted. For example:
mat

d1 $ sound "bd sd"

The first event should give the numbers ((1,1),(3,1)), to highlight the 'bd ', from column 1 to 3.

To make this more useful you have to tell tidal where each mininotation string is in the original pattern. Here's my (not quite working) attempt at doing that with javascript.

var indexToColRow = function (index, string) {
  const sub = string.substr(0, index);
  const row = (sub.match(/\n/gs) || []).length;
  const col = ((sub.match(/[^\n]+$/gs) || [''])[0].length);
  return([col,row])
}
var mungePattern = function(pat) {
  return pat.replace(/"[^"]+"/sg, function(match, index) {const [col, row] = indexToColRow(index, match); return `(deltaContext ${col} ${row} ${match}})`});
}

With that

mungePattern('d1 $ sound "bd sn" \n  # speed "2 3"')

returns

d1 $ sound (deltaContext 7 0 "bd sn"}) 
  # speed (deltaContext 5 0 "2 3"})"

I.e. it tries to add the row and column to the source positions in the pattern, which should then be reflected in the highlights you get back.

However there's something wrong with my javascript as those aren't the right col / row values! Not sure what's going on as the 'indexToColRow' function seems to work on its own.

Anyway hope that makes some sense, I'm pretty tired atm.. But this is how feedforward does it. There's a big assumption that double quotes are only used to delimit mininotation. That's generally the case.. But it will create confusing errors if you don't match your double quotes. It's a big hack really!

2 Likes
3

Thinking about it a second more.. tidal-listener could take care of this munging on the haskell side, using the existing 'deltaMini' function.

4

That makes a lot of sense, thanks! I hadn't realized that there was an extra step of injecting new metadata code, but that makes sense. I'm happy to keep tweaking your Javascript example and figuring out how to get it working.

5

I found this interesting so I did some digging and opened Reimplementing event highlighting in tidal-listener · Issue #1047 · tidalcycles/Tidal · GitHub to share my thoughts and spark a new discussion.

4 Likes
6

11 posts were split to a new topic: Pulsar Editor Event Highlighting

7

Wow, your implementation already looks amazing!

1 Like
8

Looks absolutely perfect already, thank you for doing that!
This will be extremely valuable for teaching, or just in general to keep track of what's happening in longer patterns.

1 Like
12

I'm glad my feedback was useful!

I'm doing a performance at Nø School Nevers on Friday 11th July. If the main repo (or even your fork) is available to test I'd love to try it out for the performance!

1 Like
13

Thanks for the offer, but unfortunately I will not make it in time. My attention is it to make this feature as robust as possible and I try to address some edge cases.

For this I needed to do a complete rewrite of the event handling logic (tbh twice), but it looks really promising. Who would have seen it coming that time-accurate animations are hard to implement in a JavaScript environment?! :smiley:

Anyway - I am quite optimistic, that I am able to create a merge request within the next days. Stay tuned!

4 Likes
21

Thanks to this thread, I was able to add event highlighting to my pluguzu editor. In my case it’s a bit different because the editor is in charge of submitting the tidal events based on an external clock, but in case that can be helpful to other implementation, here is how I did it:

  • First use Sound.Tidal.Pattern.deltaMini on the input source to inject deltaContext around mini-notation strings.
  • Access the list of mini-notation locations related to a given event from the Event.context.contextPosition. In my case, I only support one-line mini notation, so I only kept the row, column and length of the location.
  • I use egui for text editor interface, and it provides a convenient list of glyph (called galley) which can be used to get the pixel coordinate of a given character. For the record, it looks like this: TextEditOutput in egui::widgets::text_edit - Rust , and here is how I integrated it: highlights.rs
  • So when an event is submitted, I add the context positions to a temporary list which is used to paint highlight boxs on top of the text. egui is a immediate mode GUI library, like dear-imgui, which means that everything is drawn to the screen at the display refresh rate. This let me simply progressively dim the highlights and remove them from the temporary list after a small period. For the record, here is the drawing logic I wrote: ui.rs

This is a bit of a special case because I don’t use the tidal OSC stream, though I hope this can still be useful for other editors! Cheers