<_whitenotifier-e>
[scopehal-apps] azonenberg 78acce9 - FilterGraphEditor, WaveformArea: very early initial version of graph node deletion
<_whitenotifier-e>
[scopehal-apps] azonenberg 020b11b - MeasurementsDialog is now a SinkNode, graph deletion works better
<d1b2>
<azonenberg> Just pushed an update that makes the delete key work as expected in the graph editor for a significant subset of cases
<d1b2>
<azonenberg> It will walk the now-bidirectional filter graph, find all of the sinks it knows about, and break those links
<d1b2>
<azonenberg> We still use ref counting under the hood, which means that if anything not in the graph (e.g. a properties dialog) is consuming the output of the filter, it will have a dangling reference and not self-delete
<d1b2>
<azonenberg> but all filters, waveform views, and measurements will be disconnected
<d1b2>
<azonenberg> and if you then close the properties dialog it'll self delete
<d1b2>
<azonenberg> Not quite where I want to be, but closer
<d1b2>
<azonenberg> Making the properties dialog be a graph node will happen soon
<_whitenotifier-e>
[scopehal-apps] azonenberg 382079a - Channel properties dialogs are now tracked as filter graph nodes and will cleanly go away when the node is deleted
<d1b2>
<.artag> I'm attempting to build ngscopeclient on Debian Trixie using the 'getting started' guide. The binary seems to run fine, GPU is an old Radeon. I'm using the default vulkan from debian which is 1.3.224. At 'make' I get :[ 1%] Built target log [ 5%] Built target halshaders [ 6%] Built target xptools [ 6%] Building CXX object lib/scopehal/CMakeFiles/scopehal.dir/cmake_pch.hxx.gch In file included from /usr/local/include/vulkan/vulkan.h:89,
<d1b2>
from /usr/local/include/vulkan/vulkan.hpp:31, from /usr/include/vulkan/vulkan_raii.hpp:11, from /home/adrian/Projects/scopehal-apps/build/lib/scopehal/CMakeFiles/scopehal.dir/cmake_pch.hxx:6, from <command-line>: /usr/local/include/vulkan/vulkan_beta.h:590:11: error: ‘StdVideoEncodeH264RefMemMgmtCtrlOperations’ does not name a type 590 | const
<d1b2>
StdVideoEncodeH264RefMemMgmtCtrlOperations* pMemMgmtCtrlOperations; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ And lots of similar errors. I note that instructions for the tarfile install require some environment variables setting - is there anything similar required for the repo installation ?
<d1b2>
<dameck> Thank you so much, I was craving that
<d1b2>
<dameck> Yesterday I encountered this. I calculated a differential signal (The Blue WF and then fitted a generated Sinus to it (Salmon color almost entirely overlaying the original). Then I wanted to measure the difference and my calculation shows differences of more higher than the amplitude of individual Waveforms. Could this be some unit/scaling issues in individual waveforms?
<d1b2>
<dameck> Another question regarding the use. Let's say I have my transmitted signal and my now fitted (maybe sometime much better) expected data. Does someone from the top of their head have an idea to properly calculate the SNR (in dB) for that? afaik the current SNR filter that calculates mu/sigma is only really usefull for a constant waveform or do I missunderstand?
<d1b2>
<azonenberg> It was prboably the single most requested feature lol
<d1b2>
<azonenberg> it's just that implementing it required a LOT of backend work before we could actually track down everyone who was using the filter and break those links cleanly
<d1b2>
<azonenberg> the original data structures were not built with that kind of thing in mind and undoing that mistake from years ago was not easy
<d1b2>
<dameck> I have also noticed, that tearing stuff like a dependency graph down cleanly mostly takes more work than constructing it
<d1b2>
<azonenberg> hmm, our CI builders are using the debian packaged vulkan SDK too
<d1b2>
<azonenberg> and they build without issue
<d1b2>
<azonenberg> we need to update the docs to reflect that vulkan stopped shipping .deb packages of their own, now you have to either install from the tarball or use distro packages
<d1b2>
<azonenberg> I use the tarball on my dev machine to get the extra utilities and debug features, but the CI builders run the debian version since that is what we expect end users to be doing
<d1b2>
<azonenberg> Are they the same sample rate?
<d1b2>
<azonenberg> The subtract filter is a simple pointwise subtraction, out[i] = inA[i] - inB[i]
<d1b2>
<azonenberg> it does not resample or align the signals
<d1b2>
<azonenberg> i think it will correct for a fixed time offset if you have a deskew correction on one of them, but that's it
<d1b2>
<dameck> Ahh, I kinda expected them to be resampled to the same time unit. Yes, one is at 100MS and one at 9.XMS
<d1b2>
<azonenberg> Yep that's your problem. File a ticket and I can make it add a warning if the sample rates are mismatched
<d1b2>
<azonenberg> You can use the "upsample" and "downsample" to integer resample one waveform to match the rate of another. We do not currently support fractional resampling although it's on the wishlist
<d1b2>
<azonenberg> you can cascade them to up/down sample by any integer ratio within reason although with deep memory you'll eat a lot of ram because it has to store the whole upsampled waveform
<d1b2>
<azonenberg> (a proper fractional-N resampling block would just store that in registers and not need a whole 9x upsampled waveform of buffer to resample by 9/13 or something)
<d1b2>
<dameck> am I silly or could you implement simple fractional-N resampling using bresenheims algorithm?
<d1b2>
<azonenberg> I havent looked into how to do it, it's one of a million "would be nice" things i haven't had time to even think about because I'm putting out other fires :p
<d1b2>
<azonenberg> If we had 10x as more active contributors it would be a different story
<d1b2>
<azonenberg> Right now i'm fopcusing on a lot of the core data model rework because I'm the only one who understands all of the fine points and what needs to be done
<d1b2>
<dameck> sounds like a good getting started issue. I'll give it a try once I done some more work on my Bachelor thesis practical part
<d1b2>
<azonenberg> and documenting the current state enough that someone else could do it would be more work than just doing the refactoring and documenting the redone state
<d1b2>
<azonenberg> Yeah we have a lot of DSP work that would be nice if somebody has the skills and time
<d1b2>
<azonenberg> I want a fractional-N synthesizer PLL filter so if you give it any arbitrary frequency it can make a multiple/division of it
<d1b2>
<azonenberg> (useful for things like HDMI where you get a 1/10 rate word clock then high speed bits)
<d1b2>
<dameck> Also, when would the next zoom meeting be, I'd be interested in learning more
<d1b2>
<azonenberg> i want a CDR PLL that has specific jitter transfer curves matching standards for compliance testing, rather than our current PLL that is good enough for data recovery and basic SI analysis but doesn't match the exact transfer function of a particular standard
<d1b2>
<azonenberg> Whenever I have time to sit down and breathe to schedule one
<d1b2>
<azonenberg> we're way overdue
<d1b2>
<azonenberg> I want to do them monthly and have not had the chance
<d1b2>
<azonenberg> Usually the way I do it is i send out a whenisgood poll in the chat and give people in different time zones a day or two to respond
<d1b2>
<azonenberg> and pick the time with the least conflicts from major currently active contributors
<d1b2>
<azonenberg> then announce the selected time and drop the zoom link in the channel for anyone to join whether they responded to the poll or not
<d1b2>
<dameck> Perhaps announcement on mastodon would also find people to join :P
<d1b2>
<azonenberg> This chat is the main coordination platform for devs, I announce releases and updates and general dev progress on mastodon but that is more targeted at end users
<d1b2>
<azonenberg> ditto for my occasional linkedin posts which are mostly flashy demos and release announcements
<d1b2>
<dameck> Well, you pulled at least one from user to wannabe dev
<d1b2>
<azonenberg> @dameck for reference, most of the protocol filters handle mixed sample rates nicely
<d1b2>
<azonenberg> e.g. if you have an i2c/spi decode with varying sample rates it will happily correct for that just fine
<d1b2>
<azonenberg> most of the simple math blocks do not
<d1b2>
<dameck> are the filters implemented in ngscopeclient or in scopehal?
<d1b2>
<azonenberg> scopeprotocols, which is a separate so/dll but lives within the scopehal repo
<d1b2>
<azonenberg> ngscopeclient is purely the frontend GUI
<d1b2>
<azonenberg> the intent is that we could have multiple frontends, e.g. a touch optimized version for tablet/phone usage, in the future
<d1b2>
<azonenberg> during the opengl-to-vulkan transition period we had glscopeclient and ngscopeclient both live in scopehal-apps
<d1b2>
<azonenberg> but currently ngscopeclient is the sole actively developed/supported frontend
<d1b2>
<azonenberg> but we also want to enable things like writing your own custom test automation tools that do protocol decodes or control instruments from C++
<d1b2>
<azonenberg> or in the future python scripting or similar but that's a ways out
<d1b2>
<azonenberg> So we try to have a pretty clear front/back end split
<d1b2>
<dameck> that's nice
<d1b2>
<azonenberg> it's not perfect, there are some gotchas and things that need refactoring to make it easier to write custom tools using the API
<d1b2>
<azonenberg> and there is no API/ABI stability whatsoever yet
<d1b2>
<azonenberg> in part because we are constantly refactoring class structures to undo design mistakes that made sense when it was a single-user debug tool for my phd thesis used to automate a rigol ds1102d and some fpga/uart logic analyzers
<d1b2>
<azonenberg> but are not so great now that we're trying to decode ethernet in real time on 8 Gbps of streaming adc samples from a thunderscope :p
<d1b2>
<dameck> but haven't you achived that already? Or was that at a lower samplerate?
<d1b2>
<azonenberg> I am. but the refactoring is ongoing and there is still a lot of architectural work i want to redo
<d1b2>
<azonenberg> e.g. we are currently limited to having the filter graph refresh once per video frame
<d1b2>
<azonenberg> so with a 60 Hz monitor you cannot trigger/decode at >60 WFM/s
<d1b2>
<azonenberg> in a headless application you could run much faster
<d1b2>
<azonenberg> but the ngscopeclient rendering pipeline is a strict waterfall model
<d1b2>
<azonenberg> acquisition runs in a separate thread with a fifo, but the filter graph and rendering are strictly sequential
<d1b2>
<azonenberg> this is one of the reasons i use deep memory so much, if you go below a few megapoints you saturate framerate and aren't gaining anything by having shallower memory depth
<d1b2>
<azonenberg> i.e. you see far more of your signal decoding 50M points at 20 Hz than 100K points at 60 Hz
<d1b2>
<dameck> quick understanding question. I suppose the FlowGraphNode::ValidateChannel method is being used to display wether a filter (or other stuff) has properly configured inputs? (I assume it also does other work for other usecases, but I'm trying to understand the subtract filter for now). That is then shown in the UI in an error message. Does the UI simply walk through all flowgraphnodes and checks their states?
<d1b2>
<azonenberg> Sooo
<d1b2>
<azonenberg> ValidateChannel is another bit of legacy
<d1b2>
<azonenberg> It is used to determine if a proposed connection is legal to make
<d1b2>
<azonenberg> i.e. if the filter graph editor should show the green check or red x when drawing the line
<d1b2>
<azonenberg> it's purely procedural not data driven
<d1b2>
<azonenberg> the plan is to reframe this as a set of structured rules like must be analog, must be digital, must be uniform, must be sparse, must be same type as another input
<d1b2>
<azonenberg> etc
<d1b2>
<azonenberg> and have an accessor each filter returns that simply gives the list of rules
<d1b2>
<azonenberg> because right now there is no way for a user to know why a proposed connection wasn't allowed, or what the filter expects
<d1b2>
<azonenberg> i would rather have it be able to show a tooltip and say "input foo: accepts sparse or uniform digital waveforms, must be same sample rate as input bar"
<d1b2>
<azonenberg> The error reporting is separate, each filter's Refresh() method can call AddErrorMessage() at any time to report the problem
<d1b2>
<azonenberg> which currently is mostly or entirely used for fatal "I cannot produce an output at all" conditions
<d1b2>
<azonenberg> but is eventually going to allow you to display warnings "you might not get the behavior you expected but I'm going to try" conditions
<d1b2>
<azonenberg> I dont think it will happen in time for v0.2. but the idea is that we'll start by adding a method to query the set of rules, make the base class ValidateChannel() check against those rules but make derived classes keep the legacy implementations
<d1b2>
<azonenberg> then gradually refactor more filters to have validation be rule based, and eventually deprecate ValidateChannel outright and work entirely off the structured rules
<d1b2>
<azonenberg> welcome to rapidly evolving, old but not mature, software projects where the answer to many architectural questions is "it made sense in 2013 and we haven't undone it yet" :p
<d1b2>
<dameck> I supposed it's a miracle a project resulting from academia has made it 13 years without having be reconstructed from scratch
<d1b2>
<azonenberg> Lol. I mean probably every line of code has been rewritten at some point since then
<d1b2>
<azonenberg> To give you an idea... h/o
<d1b2>
<azonenberg> this is what the UI (scopeclient, before glscopeclient which itself predated ngscopeclient) looked like circa 2015
<d1b2>
<dameck> So far I haven't really understood Ethernet yet and in my current project was just glad my PHY pumped out RMII
<d1b2>
<azonenberg> Not currently. It probably should. or maybe more general allowing filters to generate messages of multiple severity
<d1b2>
<azonenberg> File a ticket and we can add some notes to figure out what makes sense
<d1b2>
<azonenberg> AddErrorMessage is itself very recent, we used to just have blocks log to stdout/stderr when they had problems and give no GUI feedback whatsoever
<d1b2>
<azonenberg> So it probably needs to be made more general
<d1b2>
<dameck> where is the AddErrorMessage function declared and what implements it currently?
<d1b2>
<azonenberg> scopehal/FlowGraphNode.h:222
<d1b2>
<azonenberg> it's purely unstructured text for display, there's no concept of severities or multi line messages or anything
<d1b2>
<dameck> wanted to understand it further for suggestions (:
<d1b2>
<azonenberg> and then FilterGraphEditor in ngscopeclient actually displays it
<d1b2>
<azonenberg> it's called by most filters in scopeprotocols when something goes wrong
<d1b2>
<azonenberg> But yeah, its simplistic but this was mostly adapted from stuff basically printf-ing when there were errors and this was easy to port to :p
<d1b2>
<dameck> e
Stephie- has quit [Read error: Connection reset by peer]
<d1b2>
<dameck> If there are no fundamental issues against my suggestions I'd create a PR to simply pass the tests
<d1b2>
<azonenberg> I'll look in a bit and see what makes sense. Running both side by side during the transition period makes sense
<d1b2>
<azonenberg> The other thing is, with v0.2 being released pretty soon i want to avoid doing much in the way of additional major code structure changes until we get that out the door
<d1b2>
<dameck> Especially since I would suggest having AddErrorMessage function signature seems like an appropiate concrete implementation of the suggested AddMessage
<d1b2>
<azonenberg> Yeah AddErrorMessage should be possible to implement as a trivial inline shim around a more generic API
<d1b2>
<azonenberg> It's just a matter of thinking through the new design a bit to make sure we get it right and don't have to redo it again 😛
<d1b2>
<azonenberg> It may be worth looking at what lib/log (logtools) is doing, which is what we do for console logging
<d1b2>
<azonenberg> and see if we can e.g. maybe use the severity level enums and stuff out of that rather than making new log levels for these messages
<d1b2>
<azonenberg> we don't need trace-level output from filters since that is a developer thing and having it stdout-only is OK
<d1b2>
<azonenberg> but the higher levels make sense to probably keep as is
<d1b2>
<azonenberg> then have the bulleted list formatting etc move to be something in the filter graph editor itself
<d1b2>
<dameck> the bulleted list formatting?
<d1b2>
<dameck> including fatal?
<d1b2>
<azonenberg> no. that's a "something is horribly wron we are going to crash and here's why" thing
<d1b2>
<azonenberg> which doesnt make sense for gui display
<d1b2>
<azonenberg> (sorry eating lunch and fingerpeck typing with my clean hand lol
<d1b2>
<dameck> you know you can step away from dev for lunch break?
<d1b2>
<azonenberg> yeah but over lunch i'm not on the clock for $dayjob and my daughter is also not home from school and demanding to play with daddy
<d1b2>
<azonenberg> So i try to be productive during that time
<d1b2>
<azonenberg> as it's usually all i have for dev until after she goes to bed
<d1b2>
<azonenberg> Anyway, yeah i think debug/verbose/notice/warning/error from logtools should be used as verbosity levels
<d1b2>
<dameck> okay, just wanted to make sure :)
<d1b2>
<azonenberg> keep the same enums, have a vector<whatever event structure> for output messages
<d1b2>
<azonenberg> the current AddErrorMessage string concatenates bullet character, message, newline into the std::string error output
<d1b2>
<azonenberg> We'll need to adapt the UI code to do any kind of multi-line formatting itself if we are generating more structured messages in the filters
<d1b2>
<azonenberg> Which I think is the right path forward
<d1b2>
<dameck> I have made some good progress on my users fork on that. I'd implement a first test warning for the subtraction filter. How do I get the samplerate for a WaveFormBase object?
<d1b2>
<.artag> Found it, I think (it's currently making much better progress in the build). Some residue of old vulcan install attempts in /usr/local were likely getting included instead of the distro ones
<d1b2>
<.artag> win !
<d1b2>
<azonenberg> ok great yeah if you have a mishmash of multiple SDKs that would definitely cause issues
<d1b2>
<david.rysk> I don’t like installing stuff into /usr/local in general because it tends to cause such issues
<d1b2>
<.artag> Fair point. I sometimes prefer it if I don't want to break something else. But I shouldn't then forget it.
<d1b2>
<.artag> I have a long history of failed attempts to run ngscopeclient, usually due to inadequate graphics hardware or a badly out of date OS. I did get to run a binary earlier in the year but I've always felt scopes should be 'softer' and was keen to be able to build it. This represents the first succes :).
<d1b2>
<azonenberg> Well we've been working hard to make the new-user experience easier
<d1b2>
<azonenberg> @.artag you can also try grabbing and installing a prebuilt .deb off the CI
<d1b2>
<azonenberg> although obviously if you are planning to actually develop/work on the code yourself you do want to be able to build locally
<d1b2>
<.artag> Sure, I did that and it looked good. And it motivated me to try the build again, knowing it could work
<d1b2>
<azonenberg> Yay
<d1b2>
<.artag> I'm stunned by how fast it initializes. So many bits of modern software take forever to even paint a scree, let alone be ready to run
<d1b2>
<azonenberg> ngscopeclient is built for speed from the ground up. Not that initialization was itself the goal, runtime performance once you're set up is the higher priority. but generally it's clean and low overhead
<d1b2>
<azonenberg> it will absolutely saturate your CPU, GPU, and use all the RAM/VRAM you have if you push it hard enough. but that's only because you're asking it to :p
<d1b2>
<azonenberg> not because it's running five copies of webkit and chromium to display a help menu
<d1b2>
<.artag> heh
<d1b2>
<azonenberg> but like, i pretty much never close VTune on my dev box because i'm profiling and tuning it constantly
<d1b2>
<azonenberg> No matter how fast it is, i always want more
<d1b2>
<azonenberg> but yeah if you think it starts fast wait until you're doing, like, 20M point FFTs at 50 Hz or something
<d1b2>
<azonenberg> (you do have to turn off peak detection for that, the current peak detector is very slow and inefficient and i havent had time to optimize yet
<d1b2>
<azonenberg> like it takes more time to find peaks than to do the actual FFT by a factor of ten or more
<d1b2>
<.artag> Is acquisition / transfer the worst bottleneck ? Assuming CPU/GPU can be mostly solved with money ?
<d1b2>
<azonenberg> Getting data off the scope is something we can't do anything about. So basically you either buy a thunderscope (max streaming rate but worse analog performance), a lecroy (about 10x lower transfer speed but goes out to tens of GHz analog bandwidth if you have deep pockets)
<d1b2>
<azonenberg> or a picoscope (compromise on both BW and performance)
<d1b2>
<azonenberg> Or you accept that getting data into ngscopeclient will be slow
<d1b2>
<azonenberg> 10x for lecroy is optimistic actually, more like 15
<d1b2>
<.artag> Have you looked at theb Saleae scope ? I guess it would compare with the Pico (though some picos have impressive bandwidth)
<d1b2>
<azonenberg> I do not own one, i would like someone to write a driver but am not going to buy one myself
<d1b2>
<azonenberg> I will merge one if it shows up as a PR with a compatible license
<d1b2>
<azonenberg> But i'm wearing way too many hats as is - i maintain the drivers for siglent vector signal generators, power supplies, active loads, and function generators
<d1b2>
<azonenberg> some of the picoscopes
<d1b2>
<azonenberg> pico VNA
<d1b2>
<azonenberg> mutilane BERT
<d1b2>
<azonenberg> R&S multimeters and PSUs
<d1b2>
<azonenberg> thunderscope
<d1b2>
<azonenberg> and lecroy scopes
<d1b2>
<azonenberg> and also i'm the lead dev, and do most of the GUI work, shader optimization, and core data model/API stuff
<d1b2>
<azonenberg> i can't take on more drivers even if a free scope shows up on my porch
<d1b2>
<azonenberg> oh i almost forgot i also maintain the digilent analog discovery etc driver, but i need to fix my adp3450
<d1b2>
<azonenberg> it's in pieces on my bench with a PSU issue i haven't root caused yet
<d1b2>
<.artag> I'm interested in drivers but will start low on power supplies and multimeters
<d1b2>
<azonenberg> Yeah those are easy and useful
<d1b2>
<azonenberg> And you can do a lot of useful stuff in ngscopeclient graphing meter readings over time and doing math on them
<d1b2>
<azonenberg> especially between multiple instruments, like e.g. multiplying power and current reported by a psu to get power out
<d1b2>
<azonenberg> then measuring voltage and current at the output of a dc-dc, multiplying that to get power at that point
<d1b2>
<azonenberg> subtracting to get losses in the switcher
<d1b2>
<azonenberg> dividing to get percent efficiency
<d1b2>
<azonenberg> etc
<d1b2>
<.artag> yeah. I have an SMU but it could be replaced with independent instruments
<d1b2>
<dameck> I currently even use it without a scope attached just importing csv exports from my Owon until my rigol arrives. Once it's here I'll look into getting the AFG on the 924 working
<d1b2>
<azonenberg> Awesome. yeah it's absolutely usable offline too you just dont benefit from the ludicrous speed as much :p
<d1b2>
<azonenberg> The thunderscope is so far really the only thing that can push its limits on a decent system as far as performance on the well-optimized, GPU-accelerated filter blocks