materialize JSON-LD versions of the manifests #254
Conversation
note that these versions are generated internally to generate the HTML versions. This change simply writes the results in a file. Some people might find this more convenient to process than the TTL files? I don't feel strongly about this, I ended up doing this while getting myself familiarized with the automation process.
Tpt
left a comment
Tpt
left a comment
There was a problem hiding this comment.
Nice addition for implementations that might want to run some tests without a proper Turtle parser
| %w(recognizedDatatypes unrecognizedDatatypes).each do |p| | ||
| if entry["mf:#{p}"].is_a?(Hash) && entry["mf:#{p}"]['@list'] == [] | ||
| entry[p] = [] | ||
| entry.delete("mf:#{p}") |
There was a problem hiding this comment.
It might be nice to run these normalizations inside of the JSON-LD serialization code to provide more normalized files
| "eval/manifest.ttl", | ||
| "syntax/manifest.ttl", | ||
| "../../rdf11/rdf-trig/manifest.ttl" |
There was a problem hiding this comment.
These aren't being transformed.
|
Rather than JSON-LD in the tree (FWIW they used to be produced), I think it is better to produce the jsonld files as the GH-pages are created. Then there is one version of the truth. |
I sympathize with that, and to mitigate the potential issues, I added a "comment" at the top of each JSON-LD file, explaining that they are generated and should not be edited. Arguably, the whole generation process could be restricted to the page generation, rather than materialized in the repo. Or do you consider that the HTML pages are different from the JSON-LD files? |
I saw that.
The design I see as best for the long-term is that the repo is the primary copy of the tests. There is a website that is a presentation of that content for certain audiences. It is produced from the repo files with HTML, copies of the all the files and JSON-LD manifests. |
as a nice-to-have, we should probably also release zips containing all the files, including the generated ones... (again, some people might be interested in processing the JSON-LD manifests rather than the TTL ones, and still do that from a local copy) Do you think this change (moving from "in repo generation" to "in pages generation") should be included in this PR before merging? |
Seems fairly easily alongside GitHub pages publishing. I will give it a try today |
I won't block but this PR is all about adding the JSON-LD manifests to the repo. I do not support that. |
|
MR that stops to push HTML pages but publishes them to GitHub pages: #255 |
Github provides download as zip. https://github.com/w3c/rdf-tests/archive/refs/heads/main.zip |
That's fair. If #255 is accepted, once merged, I will make a new PR with only the changes on Rakefile, so that the JSON-LD versions of the manifests will only be present on github-pages and the released ZIP.
Yes, but if we restrict the generation of HTML and JSON-LD manifests to github pages, these ZIPs will only contain the TTL manifests. Some people might prefer that, but some people may want the full set of files, including the generated ones. |
Please substantiate "some". We simply can not accommodate every possibility we can think of. Why can't we assume that users can convert using local or online tools if necessary for them? This repo contains Turtle and N-Triples tests. They are likely to have a TTL parser. The tradeoff is having one version of the truth. And it does get out of step. GH actions fail, then the newest tests are in the TTL and not the JSON-LD yet they are the focus at the time. |
|
By "release zip", do you mean the built state at a point in time with a version number? |
I have one use case: when writing the firsts NTriples and Turtle parser in Rust for Oxigraph, I would have liked a JSON version of the manifest to parse it with an existing JSON parser and get the testsuite for NTriples to validate my NTriples parser without having built my Turtle parser yet.
I think the idea of not committing the JSON-LD files but building them on the fly when publishing the website is a good workaround: if the action fails the JSON-LD files are just not generated and the website not published. |
3 participants
note that these versions are generated internally to generate the HTML versions. This change simply writes the results in a file.
Some people might find this more convenient to process than the TTL files?
I don't feel strongly about this,
I ended up doing this while getting myself familiarized with the automation process.