> Also, a more Rust-friendly model would have incured higher memory costs.
I'm not sure how modelling everything in a rust borrow checker friendly way would change anything here? Everything you're talking about doing in C could be done more or less exactly the same in rust.
Arenas are slightly inconvenient in rust because the standard collection types bake in the assumption that they're working with the global system allocator. But there's plenty of high quality arena crates in rust which ship their own replacements for Vec / HashMap / etc.
It also sounds like you'd need to write or adapt your own JSON parser. But it sounds like you might be writing part of your own JSON / Yaml parser in C anyway.
Arenas aren't the issue. Its objects with mixed lifetimes and mutability. I cant easily model the lifetimes of objects when there are cases like an instance where buffer memory reachable from the object has a different lifetime than maps/lists. Also these objects could be transively shared or mutable. In order to make a Rust friendly model, I'd have the tree be all shared or all mutable, and have all reachable memory have the same lifetime. This would often mean allocating the full tree into one arena. That is where the overhead comes from. Each arena would need enough memory to store the entire object; currently they can be smaller since they often only need to hold parts of objects, not the entire thing.
> Also, a more Rust-friendly model would have incured higher memory costs.
I'm not sure how modelling everything in a rust borrow checker friendly way would change anything here? Everything you're talking about doing in C could be done more or less exactly the same in rust.
Arenas are slightly inconvenient in rust because the standard collection types bake in the assumption that they're working with the global system allocator. But there's plenty of high quality arena crates in rust which ship their own replacements for Vec / HashMap / etc.
It also sounds like you'd need to write or adapt your own JSON parser. But it sounds like you might be writing part of your own JSON / Yaml parser in C anyway.