@sjvn I'm sure you're aware of OLMo. I've noodle with it and it's not quite baked but am glad it exists. Any others out there?
"A highly performant, truly open LLM and framework intentionally designed with access to the data, training code, models, and evaluation code necessary to advance AI and study language models collectively."
https://allenai.org/olmo