Guest64 has joined #prjcombine
Guest64 has quit [Client Quit]
Wanda[cis] has quit [Quit: Idle timeout reached: 172800s]
Wanda[cis] has joined #prjcombine
<Wanda[cis]> okay, I have migrated serialization off serde and fixed some other problems with it along the way
<Wanda[cis]> the one problem with this is that it resulted in invalidating all caches I had, so I'm still waiting for all the databases to recompute from nil
<Wanda[cis]> also, after having to fix up some fallout that resulted in broken bsdata databases several times, I have started thinking how to integrate caching into prjcombine-hammer in a way that wouldn't require excessive invalidations, particularly when something screws up the fragile prerequisites of the algorithm
<mupuf> yeah, migration of DBs will become more important as time goes by, because replicating the extraction may get harder and harder
<Wanda[cis]> yeahh, I need to actually come up with some satisfying way to make the extraction reproducible in the first place
<Wanda[cis]> right now it's not even possible to use all the crap in re/ subdirectory outside of my own machine, or at least with a copy of ISE / icecube with the same patches applied
<Wanda[cis]> of course, this is a bit of a problem given that you need to obtain a copy of the vendor tools in the first place, and some of them are ... hard to come by
<Wanda[cis]> I have absolutely no recollection of how I obtained the copy of ISE 2.1 I'm using to reverse XC5200, for one
galibert[m] has joined #prjcombine
<galibert[m]> may I suggest generating the data as a human readable text file, and building databases from that?
DemiMarieObenou4 has joined #prjcombine
<DemiMarieObenou4> I like that approach
<Wanda[cis]> this is not a viable strategy
<Wanda[cis]> prjcombine extracts a ridiculous amount of raw data from the toolchains to do its thing
<galibert[m]> Ah, that’s for a format in the middle, not the final results
<galibert[m]> Sorry, I misunderstood
<Wanda[cis]> this is with binary serialization and zstd compression
<Wanda[cis]> I also don't particularly enjoy writing text serialization and deserialization code when I can just #derive something that will roundtrip
<galibert[m]> Sure
<Wanda[cis]> it would be nice if someone invented a standard text serialization format that I can just serialize and deserialize without major pain
<Wanda[cis]> I don't really like commiting a zstd'd blob as the main source of truth into the repo, but it seems to be the easiest way to get the job done
<Wanda[cis]> I also emit the final database as a json file in parallel, which is a massive pain because json is not a good serialization format; I originally did it so that the python doc generator can read it, but it was dogshit so it got rewritten in mdbook+rust; nowadays I keep it only so git diff works
<galibert[m]> Urgh json
<Wanda[cis]> got better ideas?
<Wanda[cis]> https://github.com/prjunnamed/prjcombine/blob/main/databases/siliconblue.json here. text format. human readable, arguably.
<Wanda[cis]> and can be parsed by something that isn't Rust. if you like pain.
<galibert[m]> I have my own kinks where it comes to file formats
<galibert[m]> Heavy normalisation (to end up with things that are essentially tables) and column formats are usually my thing
<galibert[m]> They tend to be easy to fix by hand or analyze when something went wrong
<Wanda[cis]> ... are you seriously proposing I export this stuff as csv
<galibert[m]> But heh, my kink
<galibert[m]> Nope, space separation, fixed commun width if sane
<galibert[m]> Column
<galibert[m]> Check the tables in mistral if you’re bored