new cabana feedback

[adeebshihadeh]

@deanlee has been working on a rewrite of comma cabana in Qt. Starting this thread for feedback now that it's in a stable state.

original cabana: https://cabana.comma.ai
new cabana: https://github.com/commaai/openpilot/tree/master/tools/cabana

[adeebshihadeh]

Performance is already great. Some miscellaneous notes while playing around with it:

• no popup when adding a new signal. just make it with the default settings
• drag to adjust signal size
• get DBC from carParams
• separate setting for the time range on plots instead of just using the cache size. for example, I have lots of RAM, so I want to keep tons of segments in memory, but this makes the plots unusable without zooming. 180s is a good default for this I think
• sort signals on the left only by address, ignore bus
• load DBC from paste
• show signal name on hover in binary view, like cabana
• click on video to play/pause
• option to use high quality video, like the replay tool this should fall back to qlogs
• show MSB/LSB of signals in the BinaryView
• replace message edit popup: Remove msg name popup cabana#108
• add frequency column to list of messages on the left (replace the count with this?)
• use a monospace font for the data in the "Bytes" column
• search filtering should also allow searching by bus and ID
  e.g. searching 2: should show all the messages on bus 2
• warn when DBC message length is wrong

bugs

• graphs: zooming in only works on background. clicking and dragging doesn't work on the plot lines. easy to see on a "COUNTER" signal
• time column is cutoff
• enable word wrap for column headers

[jyoung8607]

Re: frequency vs. count #26105, both are useful. Count is handy when looking for Panda safety violations, UDS fingerprint transmissions, etc. If display space is constrained, maybe make that column toggle-able, I'd want both available but don't need them simultaneously.

[jyoung8607]

I'm making use of this while finishing #24012, and I just want to say I'm incredibly impressed by @deanlee's work so far. In some respects, this is already better than old Cabana! Most of the notes I have so far are UI issues.

• The drop-down DBC list needs to be sorted alphabetically.
• We should be able to resize the first and second columns. Expanding the window gives all new space to the first column, which might be useful sometimes for CAN-FD, but I'd also like to see more of the nice new spreadsheet-view at the bottom of the second column.
• In the second column, in the signal list, I would expect clicking on the right-carat and down-carat to expand and collapse the signal definitions. This doesn't work, you have to click on the name.
• In the third column, it would be very helpful if all the charts were drawn the same size. The different length Y-axis labels changes the size of the whole chart, which makes it hard to visually line up time-correlated events across different signals. This is a problem in old Cabana too. BTW allowing us to pop-out the charts in larger form is fantastic!

[MoreTore]

Feature Part 1: Add a textbox that I can type in a timestamp to go to. There should also be a button beside the textbox that says "Goto" or I can press return/enter key.

Feature Part 2: When I make a signal I want to be able see the timestamp where that signal changes. In it's simplest form I can imagine 2 buttons on each signal graph. One button could read "Prev Change" and "Next Change". When I press the buttons It would fill the textbox with the timestamp then I can press "Goto" btn when I am ready. I think it would be a good idea to think about how advance this feature could become.

Another level of complexity would be to goto next/prev timestamp where signal is ==, >, < a value. The options could be selected with a dropdown list or a new window.

Another idea that may also be easier to implement would be to automatically list all the recorded states of a signal as a histogram.

[MoreTore]

Love the find feature!

[MoreTore]

working on WSL2 windows 11!

[adeebshihadeh]

Some appreciation for the great tools @deanlee has been writing from our internal slack:

[adeebshihadeh]

Alright, here's round two of feedback @deanlee. It's got a lot of nice things that web cabana doesn't have, but there's still some things I'm missing from web cabana.

feature requests

• save DBC; can just be a popup for now where you can copy/paste
• support multiple instances using a prefix: see how process replay does it
• auto detect light/dark theme from system

bugs

• signal list: expanding a signal has a visual glitch. seems like the scroll bar is there for one frame then gets hidden?
• charts: add some margin to prevent x/y axis values being too close or overlapping
• signals: once chart is open, show a hide chart button (just an X?)
  • also make those buttons bigger. I suggest making the whole row taller
• signal container: make the whole thing behave like cabana. instead of being in a fixed-height scrollable container, the parent widget should be scrollable and the container layout should expand to whatever size it needs to

[sshane]

Nice work so far! Just some things I noticed:

• You have to play/pause to make signal search update. https://user-images.githubusercontent.com/25857203/199191358-470f69aa-d84c-4572-a620-13214f7527dd.png
  • Actually, you have to play/pause to make new graphs show their data too
• Can we add padding above and below the axes limits so you can more easily see the plot when the data is at the min/max? https://user-images.githubusercontent.com/25857203/199191750-c4f2c368-1b42-4ac9-9f72-f838be92ccde.png

[jyoung8607]

current cabana in master is applying signal labels backwards in the message breakdown. It's reading the signals right AFAIK, displaying the bit patterns in the right places, but applying the labels left-to-right instead of right-to-left. As an example, the values for MO_StartStopp_Status are correctly set (byte 1 bits 4 and 5, lsb 12 msb 13) and correctly rendered in graphs, but the label in the message breakdown is rendered over bits 2 and 3 with msb and lsb flipped. Same for MO_StartStopp_Wiederstart, etc.

[deanlee]

@jyoung8607 I'm writing a quick fix for this little endian flipping issue #26409 . there are still some other issues with the BinaryView, I'll continue to improve it after this quick fix.

[pd0wm]

Great work @deanlee!

Small complaint, maybe somebody already mentioned this. Changing signal names can sometimes be a bit frustrating because they change order, and the cursor doesn't stay with the current signal. Either the cursor needs to stick with the current signal, or order signals by start bit so the ordering is constant.

The biggest feature from web cabana I'm missing is overlapping multiple lines on the same plot. This is useful when comparing request vs actual value, or seeing if two signals represent the same physical value.

[deanlee]

Changing signal names can sometimes be a bit frustrating because they change order, and the cursor doesn't stay with the current signal. Either the cursor needs to stick with the current signal, or order signals by start bit so the ordering is constant.

fixed at #26529

[deanlee]

The biggest feature from web cabana I'm missing is overlapping multiple lines on the same plot. This is useful when comparing request vs actual value, or seeing if two signals represent the same physical value.

WIP: #26538

[gregjhogan]

some things I wish cabana had:

• option to show dots for each value on the plot (easily see if a signal is outputting a constant value multiple time steps in a row and if a signal has an irregular frequency)
• large plot area (especially in the width dimension) and the ability to look at a plot for a signal over an entire route (maybe up to 25 segments?)
• histogram of values for a signal over an entire route (maybe up to 25 segments)

[deanlee]

large plot area (especially in the width dimension) and the ability to look at a plot for a signal over an entire route (maybe up to 25 segments?)

WIP: #26575

[MoreTore]

The histogram would be very useful

[adeebshihadeh]

@deanlee let me know if you want to write something about new cabana for the 0.9 blog post. no problem if not, I can write something up

[deanlee]

Thank you @adeebshihadeh ! I have nothing to write:)

[adeebshihadeh]

blog post is up! https://blog.comma.ai/090release/#new-cabana

[MoreTore]

Thank you @deanlee for the new cabana. It's a huge improvement!

[deanlee]

I'm glad you like it:). feel free to send any feedback.

[robbederks]

So responsive! Also found a few bugs:

• If you pass in a segment name instead of a route name, cabana opens up without errors, but nothing seems to load instead of the timeline
• If you try to zoom in by dragging in a plot from right-to-left, cabana segfaults. Left-to-right works fine this also seems to segfault sometimes

[deanlee]

If you pass in a segment name instead of a route name, cabana opens up without errors, but nothing seems to load instead of the timeline

Can you give me a problematic segment name for testing? I tested some segment name myself and haven't found this problem yet.

updated:
Is it because there are only rlogs in that route? so Cabana can't read any CAN messages.

If you try to zoom in by dragging in a plot from right-to-left, cabana segfaults. Left-to-right works fine this also seems to segfault sometimes

fixed in #26598

[deanlee]

If you pass in a segment name instead of a route name, cabana opens up without errors, but nothing seems to load instead of the timeline

fixed in #27107

[MoreTore]

I am having an issue using wsl2 with the new cabana. When executing w.show(); it exits and I get a segmentation fault.

stack trace:

swrast_dri.so![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libQt5Gui.so.5!QOpenGLShaderProgram::link() (Unknown Source:0)
libQt5Gui.so.5![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libQt5Gui.so.5!QOpenGLTextureBlitter::create() (Unknown Source:0)
libQt5Gui.so.5!QPlatformBackingStore::composeAndFlush(QWindow*, QRegion const&, QPoint const&, QPlatformTextureList*, bool) (Unknown Source:0)
libQt5XcbQpa.so.5![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libQt5Widgets.so.5!QApplicationPrivate::notify_helper(QObject*, QEvent*) (Unknown Source:0)
libQt5Widgets.so.5!QApplication::notify(QObject*, QEvent*) (Unknown Source:0)
libQt5Core.so.5!QCoreApplication::notifyInternal2(QObject*, QEvent*) (Unknown Source:0)
libQt5Gui.so.5!QGuiApplicationPrivate::processExposeEvent(QWindowSystemInterfacePrivate::ExposeEvent*) (Unknown Source:0)
libQt5Gui.so.5!QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) (Unknown Source:0)
libQt5Gui.so.5!QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
libQt5XcbQpa.so.5![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libglib-2.0.so.0!g_main_context_dispatch (Unknown Source:0)
libglib-2.0.so.0![Unknown/Just-In-Time compiled code] (Unknown Source:0)
libglib-2.0.so.0!g_main_context_iteration (Unknown Source:0)
libQt5Core.so.5!QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
libQt5Core.so.5!QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (Unknown Source:0)
libQt5Core.so.5!QCoreApplication::exec() (Unknown Source:0)
main(int argc, char ** argv) (\root\openpilot\tools\cabana\cabana.cc:31)

if I remove w.show();, the program runs. plotjuggler works but so my display is configured. I have the newest Nvidia drivers on windows 11.

[MoreTore]

the launch.json file that I used for the trace.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [

        {
            "name": "(gdb) Launch",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/tools/cabana/_cabana",
            "args": ["--demo"],
            "stopAtEntry": false,
            "cwd": "${fileDirname}",
            "environment": [
                {
                "name": "LD_LIBRARY_PATH",
                "value": "${workspaceFolder}/opendbc/can:$LD_LIBRARY_PATH"
                }
            ],
            "externalConsole": false,
            "MIMode": "gdb",
            "miDebuggerArgs": "",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                },
                {
                    "description":  "Set Disassembly Flavor to Intel",
                    "text": "-gdb-set disassembly-flavor intel",
                    "ignoreFailures": true
                },
                {
                    "description": "ignore SIGUSR2 signal",
                    "text": "handle SIGUSR2 nostop noprint pass"
                }
                
                
            ]

        }


    ]
}

[jyoung8607]

WSL2 is a little moody about working with all the openpilot QT tools (plotjuggler, cabana, UI).

Try launching with LIBGL_ALWAYS_SOFTWARE=1 and see if that helps.

[MoreTore]

(openpilot-py3.8) root@MSI:~/openpilot# LIBGL_ALWAYS_SOFTWARE=1 ./tools/cabana/cabana --demo works. Thank you

[sshane]

Streaming is a little janky still. If you click somewhere on the graph, the timeline marker will constantly lag behind. Possibly fine for some use cases, but would like the reset view button to fix this.

Also, the spark lines are delayed by a few seconds when streaming.

[deanlee]

This is because streaming mode also supports seeking and zooming. you can click on the blank space on the right to reset the time to the latest( the sparkline will reset to the latest too):

There really needs to be a clearer way to do this. like a skip-to-end button

[sshane]

Can we print why it failed to load a DBC from clipboard? It shows the reason when you try to load the file.

[sshane]

Can we also show which ECU a message is from in the message view? And possibly which ECUs each signal is read by when expanding them

[deanlee]

Can we also show which ECU a message is from in the message view? And possibly which ECUs each signal is read by when expanding them

#29897

[sshane]

Can we write a test to open every opendbc DBC? This Honda one doesn't load

[deanlee]

I'll write a test case for this.

How should we handle this type of error? According to the DBC specifications, there should be no “ in the comment : http://mcu.so/Microcontroller/Automotive/DBC_File_Format_Documentation.pdf

char_string: an arbitrary string consisting of any printable charac-
ters except double hyphens ('"')

comment = 'CM_' (char_string

[sshane]

Fixing the DBC is an appropriate solution. Can we combine the cabana DBC parsing logic with the opendbc one?

[sshane]

I tried to use the "Find Signal" option while live streaming to find a button in the car, but it didn't seem to work, or wasn't very intuitive. I typed what value I expected, held the button, then pressed enter a bunch of times. I then let go and set the value to 0, then pressed enter again. I repeated this several times but the possible signals never went down, and it never found the button.

Is this how it's supposed to work?

[sshane]

Would be really nice to make it like a cheat engine search, press button, perform an action, press button, re-do action, etc. Like can_print_changes.py but you can keep going to narrow it down. Also similar to find_similar_bits except you don't need an existing signal

[sshane]

Hmm, high values cause weird graph behavior

[deanlee]

fixed: #29924

[dzid26]

It would be cool to have a switch (in the GUI or somewhere) for Panda to turn on receiving mode (NO_OUTPUT instead of SILENT).
https://github.com/commaai/openpilot/blob/master/tools/cabana/streams/pandastream.cc#L50

[sshane]

• Cabana often crashes when switching DBC after editing a few/lots of signals
• Can we reduce the min width of the signal search area? It prevents me from pressing Alt+Left or Right to make it use half the screen, which is useful when comparing signals

[deanlee]

So far I haven't been able to reproduce this issue. It might be better to have more detailed information, such as the information in the coredump file.

[sshane]

Posting these images since I wasn't attached in GDB when it happened

[deanlee]

this information is so helpful!. fixed this bug in #30227

[sshane]

thanks!

[sshane]

Is it normal for certain instances of Cabana to use a few GB while others are a few hundred MB?

[sshane]

Think it happened again, not sure how to get info on it. Let me know if there's anything you want me to do.

Skipped frame 125
Drawing same frame twice 125
Drawing same frame twice 125
Drawing same frame twice 125
Drawing same frame twice 125
Drawing same frame twice 125
Drawing same frame twice 125
Drawing same frame twice 125
stopping stream thread
"paused... at 5.74 s"
^C

[deanlee]

looks like a curl issue in replay. the qlog parsing thread will block the main thread on exiting.

by the way. It would be nice if #29849 could be merged. cabana cannot use the download cache without this fix.

[sshane]

So far haven't noticed this happening!

[sshane]

@deanlee this can still happen:

batman@workstation-shane:~$ sudo strace -p 3231974
strace: Process 3231974 attached
futex(0x20bb6d0, FUTEX_WAIT_PRIVATE, 0, NULL

[deanlee]

there is a deadlock in CameraWidget::stopVipcThread.

the vipc_thread->wait(); will block the UI thread, if the VIPC thread has already emitted vipcThreadConnected and wait for return, the deadlock occur. (refer to "The dangers of BlockingQueuedConnection" in this article: https://woboq.com/blog/how-qt-signals-slots-work-part3-queuedconnection.html)

#30397 can solve this issue

[sshane]

• DBC parsing error message is different if you load from clipboard or from file (try on bad DBC)
• Write a test to load all opendbc DBCs

[jakethesnake420]

@deanlee Feature request. Would be nice to have a toggle to switch between looking at can and sendcan. This is useful when running controlsd after modifying a car port can function.
this is a really simple modification.

[deanlee]

think you can do it by filtering the bus column:

[sshane]

128 is still a read back all the way through panda and boardd. I could see it being useful for when that doesn't work properly (GMCAN's logging for example is broken). You can also check for dropped sent messages

[jakethesnake420]

Maybe I'm mistaken but I am trying to modify a car port can message creation and view the changes instantly. So i use replay tool to run through various routes and run the modified controlsd.py. In order to see the changes I made I need to look at 'sendcan' since boardd is not running on my pc

I don't see any way to put 'sendcan' into 'can' without physically sending the data to a panda.

[jakethesnake420]

Feature request. I would like the ability to be able to copy my selection from the history log view like this:

Ideally if i copy values, it would add a comma seperator behind each value in the same row and a \r if its a new column.

I am able to use PowerToys OCR text selector to do it though but it only works for rows or columns but it doesn't format it properly if i do rows and columns at the same time

[jakethesnake420]

Here is the basic implementation but its not quite working right.

void LogsWidget::onSelectionChanged() {
  selectionModel = logs->selectionModel();
  QModelIndexList selectedIndexes = selectionModel->selectedIndexes();

  QString selectedText;
  for (const QModelIndex &index : selectedIndexes) {
      QVariant data = model->data(index, Qt::DisplayRole);
      selectedText += data.toString() + ", "; // Tab-separated values
      if (index.column() == model->columnCount() - 1) {
          selectedText.chop(1); // Remove the last comma
          selectedText += "\n"; // New line for new row
      }
  }

  QClipboard *clipboard = QApplication::clipboard();
  clipboard->setText(selectedText);
}

[jakethesnake420]

https://github.com/commaai/openpilot/pull/30586/files

[jakethesnake420]

Feature Request: In the loghistory widget it would be nice to be able to add complex logical filters. There could be a button labled "Complex filter" which would enable the ability to add more filters and then you can use a logic expression to specify which boolean expressions for the filters.

Each filter would be given an index. The user would then type in a logical expression in a text box like "(Filter1 and Filter2 and Filter3) or (Filter4 and Filter5 and Filter6)" or another way would be to use symbols and the filter index number like this "( 1 | 2 | 3 ) & ( 4 | 5 | 6 )".

This would require that the filter persists through the session and can be loaded again after restart. Might be a little complicated and may not be that useful but it's an idea.

[nworb-cire]

I've been getting segfaults lately on M1 macos whenever cabana enters segment 16 or later of a route. Segments 0-15 have no issues. It happens either when I let the stream play through the end of segment 15, or if I launch the segment directly by appending --16 (or some number greater than 16) to the route ID.

Log from one such incident: segfault.log

[deanlee]

fixed: #31902

[pd0wm]

Looks like the surpress highlight breaks after seeking. All signals are highlighted again.

[deanlee]

fixed: #31912

[sshane]

@deanlee it would be nice if Shift+Scrolling in the message view made it scroll sideways, like web browsers operate!

[deanlee]

done #31929

[sshane]

if you specify a segment with /7 it only loads that one segment

[deanlee]

use /7: to load segments start from 7 to the end

[sshane]

--7 works, I didn't expect /7 to differ in behavior we can leave it as is, --7 is a bug (though one I think is nice)

[sshane]

@deanlee zooming seems to hang cabana, and the timeline seeker doesn't update until the segment fully downloads (it used to update immediately)

cabana.zooming.slow3.mp4

[deanlee]

@sshane It looks like if you zoom into a very small time range, the blocking of camera_server_->waitForSent(); becomes very noticeable. Does this issue still exist if you use the --no-vipc flag? (cabana --demo --no-vipc)

openpilot/tools/replay/replay.cc

Lines 436 to 438 in b98ea81

	if (camera_server_) {
	camera_server_->waitForSent();
	}

Also, if we increase the minimum zoom range to, say, above 500ms (or 100?), instead of the current 10ms, do you think it's ok? A 10ms range feels too small to be meaningful. Within such a short time frame, it's unlikely that the vertical timeline in chart will have the chance to update.

[deanlee]

and the timeline seeker doesn't update until the segment fully downloads (it used to update immediately)

Before, the timeline was updating immediately after seeking, even if the corresponding segment wasn't loaded, which was a bug. This caused various issues like incorrect count/freq calculations. Now, the timeline updates to the seek position only after the segment is loaded. While this fixes the bug, it creates inconsistency in user experience. My plan is to keep updating the timeline immediately as before but to perform the real seek operations only after the loading completes. This should resolve the issue.