Character Export (.json?)

[4sfaloth]

I believe it would be nice to have a character export feature in an easily readable format (eg: .json or .xml) so that people could import them in other apps and easily parse it.
Is there anything like that in the backlog?

Great work so far btw!

[datdamnzotz]

There is a debug window that shows the character information in json. see the bottom of the screen for the little bug, along with the cloud with a down arrow in it, development download / pretty print of your orcbrew.

In order for the export your character to work properly, make sure you are on the edit screen.
#194 addresses this.

[4sfaloth]

@datdamnzotz thanks a lot for your quick reply.
I clicked the bug and indeed the text in there seems to have all the character information, but is it actually .json, or something else? I tried a few different json parsers (eg: notepad ++ JSON viewer plugin) and they all complained about malformed contect. I am guessing maybe the tags starting with ':' are an issue?

I could not see what you meant with

along with the cloud with a down arrow in it, development download / pretty print of your orcbrew.

should I be able to download this data in a prettier format using that 'cloud' button? Can you tell me where to find it please, since I can't seem to see it? :\

thanks in advance

[RobinLefebvre]

The format being used for data storage on the site is EDN, which you should be able to learn more about on here. We usually refer to this format as 'orcbrew'.

A member of our community has build these Go programs to make parsing between orcbrew and JSON, which you'll be able to find here

EDIT : I'll add an image with the buttons that Zotz was talking about, which are used for "pretty-printing" content.

[datdamnzotz]

there are two windows once you click the debug icon, the edn is on the left, and the json is on the right. It isn't pretty because it contains database ID's but all the info for the character is there...

[4sfaloth]

@datdamnzotz
this is very weird... I do not seem to have that cloud button, nor the second windows with the .json. See a screenshot from my browser below

In the meantime I managed to generate the .json from the EDN file using the program written by jnwhiteh, as linked above.
It seems like I misinterpreted the file's contents though. I was hoping to get all the information, in the character sheet, but it seems like it has the "decisions" made during character processing. For instance I was hoping to get final character stats, but instead it has the initial stats + all feats, race decisions etc required to compute the final stats. Is there any way to get a file with the "final" information that is presented in the character sheet?

Thanks a lot

[codeGlaze]

Are you using orcpub2 instead of https://dungeonmastersvault.com?

[4sfaloth]

Are you using orcpub2 instead of https://dungeonmastersvault.com?
indeed I was, sorry for the confusion. On https://dungeonmastersvault.com I see both buttons correctly. Seems like this is where everything's happening, guess I'll have to import my characters over from orcpub2 :)

About the second part of my last comment:

I was hoping to get all the information, in the character sheet, but it seems like it has the "decisions" made during character processing. For instance I was hoping to get final character stats, but instead it has the initial stats + all feats, race decisions etc required to compute the final stats. Is there any way to get a file with the "final" information that is presented in the character sheet?

Am I out of luck or is there any way to achieve this?

[codeGlaze]

For what you are looking for doesn't exist.

We do not have an export set up for other people's data structures. (Which is what you're secretly hoping for even if you didn't know it yet! 🤣)

[4sfaloth]

For what you are looking for doesn't exist.

We do not have an export set up for other people's data structures. (Which is what you're secretly hoping for even if you didn't know it yet! 🤣)

I understand. Thanks for all the help!

I'm not really at ease with html/js but I'll take a look at the code and see if I come up with something; if I manage to do it in a not terribly awful way I'll consider opening a PR in case anyone else finds it a useful feature

[codeGlaze]

Reopening:

Enhancing the site for JSON/XML output in known formats would be a potential "nice to have".

[RobinLefebvre]

We do not have an export set up for other people's data structures.

Might also be a good thing to start having a proper look at. I'm not certain we already have a ticket open for that, nor steps setup for how to achieve it. It'd be tough, but we do have the ability to display the final data and print it down on PDF, I don't see why we couldn't export it as well, given some effort.

Good starting point would be knowing which types of exports people'd like.

[4sfaloth]

So I peeked around the code a bit, and it seems to me the information I want is on src/cljc/orcpub/dnd/5e/character.cljc. I am not familiar with this extension (I will try to read-up a bit in the following days) but it seems like it has the character class definition with all final values. Is this right? If so I'll try to find a way to access it and expose it in .json format for instance (unless someone suggests something better of course :)). If anyone can give me some extra hints they will surely come in handy and be appreciated

(let me know if this is not the appropriate channel for this discussion)

[codeGlaze]

It's clojure/script.
Here's a wiki on the cljc portion of the languages. https://github.com/clojure/clojurescript/wiki/Using-cljc

Super busy atm, I'll try to get back to this tonight.
Feel free to join the discord. https://discord.gg/V2JjxY

[datdamnzotz]

The actual portion of the debug window runs here

orcpub/src/cljs/orcpub/dnd/e5/views.cljs

Lines 1050 to 1060 in 6ddabe9

	(if @expanded?
	[:textarea.m-t-5
	{:read-only true
	:style debug-data-style
	:value (clj->json {:browser (user-agent/browser)
	:browser-version (user-agent/browser-version)
	:device-type (user-agent/device-type)
	:platform (user-agent/platform)
	:platform-version (user-agent/platform-version)
	:character (char/to-strict @(subscribe [:character]))})}])
	])))

It is subscribing to the current character loaded (via the edit window) probably could be re-worked to pull a character or other stats, just need to point it to the right edn objects...

make-spec receives the built character object and makes the pdf's here:

orcpub/src/cljc/orcpub/pdf_spec.cljc

Lines 491 to 581 in ac75193

	(defn make-spec [built-char
	id
	{:keys [print-character-sheet?
	print-spell-cards?
	print-prepared-spells?
	print-large-abilities?] :as options}]
	(let [race (char5e/race built-char)
	subrace (char5e/subrace built-char)
	abilities (abilities-spec
	(char5e/ability-values built-char)
	(if (not print-large-abilities?)
	"-mod")
	false)
	ability-bonuses (abilities-spec
	(char5e/ability-bonuses built-char)
	(if print-large-abilities?
	"-mod")
	true)
	saving-throws (set (char5e/saving-throws built-char))
	unarmored-armor-class (char5e/base-armor-class built-char)
	ac-with-armor-fn (char5e/armor-class-with-armor built-char)
	all-armor-inventory (mi5e/equipped-armor-details (char5e/all-armor-inventory built-char))
	equipped-armor (armor5e/non-shields all-armor-inventory)
	equipped-shields (armor5e/shields all-armor-inventory)
	all-armor-classes (for [armor (conj equipped-armor nil)
	shield (conj equipped-shields nil)]
	(ac-with-armor-fn armor shield))
	max-armor-class (apply max all-armor-classes)
	current-ac @(subscribe [::char5e/current-armor-class id])
	levels (char5e/levels built-char)
	classes (char5e/classes built-char)
	character-name (char5e/character-name built-char)
	con-mod (es/entity-val built-char :con-mod)
	total-hit-dice (->> levels
	vals
	(reduce
	(fn [levels-per-die level]
	(update levels-per-die (:hit-die level) (fnil #(+ % (:class-level level)) 0)))
	{})
	(sort-by key)
	(map #(str (val %) "x(1d" (key %) "+" con-mod ")"))
	(s/join "\n"))
	speed (speed built-char)]
	(merge
	{:race (str race (if subrace (str "/" subrace)))
	:alignment (char5e/alignment built-char)
	:class-level (class-string classes levels)
	:background (char5e/background built-char)
	:prof-bonus (common/bonus-str (es/entity-val built-char :prof-bonus))
	:ac current-ac
	:hd total-hit-dice
	:initiative (common/bonus-str (es/entity-val built-char :initiative))
	:speed speed
	:hp-max (es/entity-val built-char :max-hit-points)
	:passive (es/entity-val built-char :passive-perception)
	:other-profs (other-profs-field built-char)
	:personality-traits (s/join "\n\n" [(char5e/personality-trait-1 built-char) (char5e/personality-trait-2 built-char)])
	:ideals (char5e/ideals built-char)
	:bonds (char5e/bonds built-char)
	:flaws (char5e/flaws built-char)
	:backstory (char5e/description built-char)
	:character-name character-name
	:character-name-2 character-name
	:xp (char5e/xps built-char)
	:player-name (char5e/player-name built-char)
	:age (char5e/age built-char)
	:height (char5e/height built-char)
	:weight (char5e/weight built-char)
	:eyes (char5e/eyes built-char)
	:skin (char5e/skin built-char)
	:hair (char5e/hair built-char)
	:image-url (char5e/image-url built-char)
	:image-url-failed (char5e/image-url-failed built-char)
	:faction-image-url (char5e/faction-image-url built-char)
	:faction-image-url-failed (char5e/faction-image-url-failed built-char)
	:faction-name (char5e/faction-name built-char)
	:print-character-sheet? print-character-sheet?
	:print-spell-cards? print-spell-cards?}
	(attacks-and-spellcasting-fields built-char)
	(skill-fields built-char)
	abilities
	ability-bonuses
	(save-bonuses built-char)
	(reduce
	(fn [saves key]
	(assoc saves (keyword (str (name key) "-save-check")) (boolean (key saving-throws))))
	{}
	char5e/ability-keys)
	(traits-fields built-char)
	(equipment-fields built-char)
	(spellcasting-fields built-char print-prepared-spells?))))

Probably could just reuse alot of what is going on there then toss it to the json processor.

[4sfaloth]

@codeGlaze @datdamnzotz thanks a lot for your inputs. I believe I know enough to get started now, once I read up a bit on clojure :P
Most likely I won't be able to do anything this week but next week I'll try to fork the repo and get this going

[barefootcoder]

In case anyone stumbles across this in a search like I just did ...

A partial solution/workaround that I have found is to use uswitch/ej to convert the EDN to JSON. This works well with some .orcbrew files (i.e. either ones you've downloaded from external sources, or created using the "Export" button in the app), as long as you observe the following caveats:

• If the file starts with a {, just pipe it directly through ej.
• If the file starts with binary 0357 (a.k.a. ASCII 239, a.k.a. ï), throw away the first 3 chars, then pipe to ej.
• If the file starts with ; or #, I haven't figured out how to make it work yet. :-(

Hopefully that's helpful to someone, and/or hopefully someone will tell me how to workaround that 3rd bullet point ...

[datdamnzotz]

In case anyone stumbles across this in a search like I just did ...

A partial solution/workaround that I have found is to use uswitch/ej to convert the EDN to JSON. This works well with some .orcbrew files (i.e. either ones you've downloaded from external sources, or created using the "Export" button in the app), as long as you observe the following caveats:

There is a native clojure converter already that works. Issue is just getting the right data that the export wants : ]
https://github.com/mmcgrana/clj-json

[4sfaloth]

I'm afraid I'm going to give up on this. I gave it a try but clojure is just too different from all the programming languages I know, and it's taking forever to understand what's going on and get anything done. I believe the information I want is stored on the built-char variable and so I only needed to export that as json, but I can't seem to progress any further. Hopefully someone with a better understanding of clojure / clojure script will come along and get this done :\

[barefootcoder]

There is a native clojure converter already that works. ...

Well, sure, but:

I gave it a try but clojure is just too different from all the programming languages I know, ...

Yeah, that. :-/

Currently what I'm doing is just leaving the files for classes et al in their native EDN format and editing them by hand (well, in vi). I never edit things inside DMVault any more. This offers two advantages:

• I never have to export from DMVault, so I never have to worry about what bizarre things are being done to the data on the way out.
• I never have to worry about losing my work because DMVault did something weird.

Of course, this doesn't work for everything:

• Characters is the big one. There's no real way to export characters at all. You can get a JSON dump of them via the debugging button, but that doesn't do you but so much good, since you can't use that JSON to get them back in.
• Magic items/custom weapons (which DMVault doesn't really distinguish between) don't appear to be importable. Or exportable.
• A few other minor bits and bobs.

As always, corrections and/or suggestions welcomed.

[codeGlaze]

I never have to worry about losing my work because DMVault did something weird.

Huh? DMV (OP project) is literally what gives the EDN files their structure. What weird things would it be doing?

Importing via JSON seems kind of low-pri to me. But exporting to it could have its merits.

@4sfaloth I skimmed, what template are you imagining targeting?

[4sfaloth]

@barefootcoder @codeGlaze
thank you for your interest in my issue.

I believe there is some confusion about what I was trying to achieve, and since @codeGlaze also requested it I'll try to explain myself a bit better.

I have been working on a web app of my own which I use as a DM screen on a tablet. I now wanted to be able to import characters built on orcpub into it. Mind that I am talking about built characters, so the general orcpub export is not suitable for me.
Just to make myself clearer, I have read the explanation on the orcpub design decision (https://github.com/Orcpub/orcpub#orcpub-fundamentals) and understand perfectly that it internally stores characters as a series of "decisions". This makes sense, because orcpub needs to be able to add new decisions, revert and/or change previous decisions and be able to easily build the new character.
Obviously this is not suitable for third party apps (such as mine), since they would need to implement all the 'build' logic (ie: from the decisions' list compute the final stats, etc...).
So what I was trying to export is the built char (I think on the code it's the buit-char variable) which should simply have the final stats and abilities of the character.
The file format is not that important since I guess I could always convert it later, but json would be preferred

Please let me know if my explanation is still too confusing :\

[barefootcoder]

@codeGlaze:

I never have to worry about losing my work because DMVault did something weird.

Huh? DMV (OP project) is literally what gives the EDN files their structure. What weird things would it be doing?

Well, that comment was weighted more towards editing classes, races, backgrounds, etc., where there is already an import and an export. Although that doesn't really apply to characters, which is what the OP was originally discussing, I personally got here by searching for how to convert the EDN .orcbrew files to JSON. No doubt others may arrive by the same method. So, while this specific tangent is not helpful for the OP, I hope it might be helpful for others.

AFA "what weird things would it be doing," they come in two flavors:

• If you import a file and then export it again, DMV reformats (and occaisionally reorders) it. Whether or not you consider this "weird" admittedly depends on your expectations, but it's at least annoying.
• If you edit things in DMV, you occasionally lose your work. It doesn't happen too often, happily, but the Discord contains several sad stories, and I've personally had it happen as well. In particular, DMV hates the back button. :-(

Importing via JSON seems kind of low-pri to me. But exporting to it could have its merits.

(Getting back to characters now, which is the point of the OP's question.) Well, you can already export to JSON, sorta-kinda. The reason I think importing is important is because, again, it is possible to lose your work. Or suppose you lose your password and can't get back into your account. Or suppose you just accidentally hit a "delete" button somewhere.

In those cases, the proper answer (as always), is that you should have a backup. Except, with DMV, that's problematic:

• You can save character PDFs, but that isn't enough information to fully recreate the character.
• You can get the character as JSON, even if it is a giant PITA, but there's no easy way to just suck that JSON back in. You have to open up the JSON in an editor and painstakingly recreate the character by hand.

Having a way to import characters—whether JSON or even EDN—would solve those issues.

@4sfaloth:

I now wanted to be able to import characters built on orcpub into it. Mind that I am talking about built characters, so the general orcpub export is not suitable for me.

I was assuming you knew about the debugging button, but perhaps you don't? Bring up your character in DMV and scroll to the very bottom. In the bottom right, you should see two tiny, yellow icons. The upper one looks like a cloud with a downard-pointing arrow in it. The one underneath that is supposed to be a bug, I think ... it's so tiny it's hard to tell. If you hover over it, it says "Development – Debug info". Click that. It may seem to do nothing, but you should notice now that you can suddenly scroll down even further. There you'll find two panes with your built character info in JSON. The one on the right is pretty-printed, but I'm pretty sure they're identical otherwise.

If you didn't know about that, give it a try and see if that helps solve your issue. If you did know about that, maybe you could explain why what's there is insufficient for your needs.

[datdamnzotz]

Messing around with the possibilities of exports, I discovered this in the routes. HItting
https://www.dungeonmastersvault.com/dnd/5e/characters/17592188952026
gives you the edn.

You can also access the character info from the local storage when hitting the view page.
https://www.dungeonmastersvault.com/pages/dnd/5e/characters/17592188952026

[4sfaloth]

Having a way to import characters—whether JSON or even EDN—would solve those issues.

I know for sure there is a way to do that. I did that in order to import characters over from the original orcpub server back to here. I can't recall exactly how it worked but it was kinda "fishy"; I think you had to actually open the browser console and copy some stuff and then post them somewhere else. I'll see if I can find that reference for you

I was assuming you knew about the debugging button, but perhaps you don't?

I knew about the button, but was unaware it displayed data for the built character. I thought it was just orcpub internals. I "exported" a .json character that way, it does look like it has many of the stuff I needed (namely hitpoints and final attributes), but I don't see many others like armor class or passive perception. Should all this data be visible from the debug windows?

[4sfaloth]

Messing around with the possibilities of exports, I discovered this in the routes. HItting
https://www.dungeonmastersvault.com/dnd/5e/characters/17592188952026
gives you the edn.

You can also access the character info from the local storage when hitting the view page.
https://www.dungeonmastersvault.com/pages/dnd/5e/characters/17592188952026

But that's the "internal format character", not the built one, right?

[barefootcoder]

I "exported" a .json character that way, it does look like it has many of the stuff I needed (namely hitpoints and final attributes), but I don't see many others like armor class or passive perception. Should all this data be visible from the debug windows?

You know what? I've never looked to see how much of the "calculated" info is in that JSON. Since I've only ever been using it as a backup for last-resort character re-creation, I never needed that sort of stuff before. Passive perception is trivial to calculate, of course, but armor class ... yeah, that's a PITA.

Well, if the whole manual process is not too awful for you, why not add another step? :-) What I've switched over to is saving the PDF, extracting the field values from that, then twiddling them and creating a new PDF. For your purposes, you could stop once you get the values. While this is not sufficient for what you need (I think) on its own, perhaps in combination with the debug JSON, and perhaps a little creative scripting, you could manage to cobble something together?

I personally am using pdftk to generate the .fdf file from the .pdf, but then I'm on Linux so that's very convenient. But I'm pretty sure there's a Windows version as well, or probably other products that can do this for you too.

Once you get the FDF, you can either find a product or programming library to read it for you (I'm using PDF::FDF::Simple in Perl), or just open it up with an editor and extract things by hand. Or split the difference and write a stupid-simple parser to get out what you need. There's some garbage at the top you can ignore, but all the values look the same. Here's a sample:

<< /T (ac) /V (13) >>
<< /T (acrobatics) /V (+3) >>
<< /T (acrobatics-check) /V (Off) >>
<< /T (age) /V () >>
<< /T (alignment) /V (Chaotic Good) >>
<< /T (allies) /V () >>
<< /T (animal-handling) /V (+2) >>
<< /T (animal-handling-check) /V (Off) >>
<< /T (arcana) /V (0) >>
<< /T (arcana-check) /V (Off) >>

You could probably even do that with a one-liner, certainly in Perl, or maybe even in Ruby or Python. The only bad part is that you just have to read the files for a while to know what the names are for things. And also that some of the larger boxes on the character sheet have been shuffled around, but nobody updated the code that touches the form fields, so you occasionally have weird stuff like features-and-traits contains the equipment and magic items. :-/ But it's usable. (I know, on account of I'm using it. :-D )

[datdamnzotz]

But that's the "internal format character", not the built one, right?

https://www.dungeonmastersvault.com/dnd/5e/characters/17592188952026
It's the as-built in EDN format for that specific character ID

[datdamnzotz]

@barefootcoder @4sfaloth what format you looking to export to? Just a simple dump? or roll20? or???
I have a simple dump working, (still sorting out the spells list)
https://cdn.discordapp.com/attachments/483272337798725652/696584392759312424/character-export.json

If anyone has the roll20 format @ me so I can incorporate it. Seems to be the defacto standard to shoot for.

[4sfaloth]

Sorry for taking some time to reply.

@barefootcoder @4sfaloth what format you looking to export to? Just a simple dump? or roll20? or???
I have a simple dump working, (still sorting out the spells list)
https://cdn.discordapp.com/attachments/483272337798725652/696584392759312424/character-export.json

If anyone has the roll20 format @ me so I can incorporate it. Seems to be the defacto standard to shoot for.

.json or .xml simple dump would be perfect for me, since I intend to write my own parser anyway.
I took at the file you attached in discord and it seems to be exactly what I need! There is some stuff encoded as string that would be nice to encapsulate in different fields (eg: Profficiencies by type, features and traits, etc...) but I suppose that perhaps orcpub provides that information already as a concatenated string?

At any rate this would be most useful to me! Are you going to get that dump-mechanism merged into master?

[datdamnzotz]

Still working on it ; ]
https://github.com/datdamnzotz/orcpub/tree/exports

[4sfaloth]

@datdamnzotz I just used orcpub yesterday and it has this feature now! Not sure if you had anything to do with it but it seems to be working fine and to address my request. I'm closing this :)

Thanks to whoever made this happen!

[barefootcoder]

@datdamnzotz

what format you looking to export to? Just a simple dump? or roll20? or???

My primary goal (which I believe is different than the OP's) is to be able to use a data dump to be able to recreate the character, in case of data loss. The ultimate embodiment of this would be for it to be paired with an import feature, so that both backing up and restoring are both just a click. But obviously that's a more pie-in-the-sky type of idea.

In the meantime, just a way to capture all the decision points so that, by referring to the data dump, I can at least recreate the character manually. I just had an occasion to try out the new feature, and, while it captures the vast majority of the data (aside from spells, which you mentioned wasn't done yet), I did notice a few things missing (most notably race and subrace; perhaps because it's a custom race ... ?). But maybe I should start a separate issue for that.