When using the text field of the Data Forms, the data is not always stored correctly. As far as I know, this only applies to the word “\n”.
When the user types in the following into a Data Form text field:
Anybody\nobody has reported this bug before.
This is a new line!!!
When presenting this data, it presents like this instead (live example):
Anybody
obody has reported this bug before.
This is a new line!!!
This doesn't happen in one-line input. Only multiline input.
Solution
Escape “\n” before storing the user's input into the Wikidot database.
Yes. This becomes evident when using Data Forms to submit source code.
Many programming languages interpret "\n" as the newline character, so it's likely to be entered into a data form at some point.
At the moment this means that part of the source code is missing when the page is viewed. Instead, this should be escaped as mentioned above.
Please also make sure this works for 'raw' fields (i.e. %%form_raw{text}%%).
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
The library we use for YAML parsing/decoding is seriously broken…
Piotr Gabryjeluk
visit my blog
That sounds serious…
OK, I checked this again, and it seems not so very broken as I though before, but there is a bug.
Reported: http://trac.symfony-project.org/ticket/9827
If they fix this fast, we'll just update the library.
Piotr Gabryjeluk
visit my blog
I imagine if they don't fix it soon (whatever “soon” means), that you will provide some sort of internal YAML patch.
Curiously, for only you would know, does this only apply to the word “\n” or are there other glitchy words?
I actually fix the problem with the library and attached a patch to the bug so they can apply it to their code. If they don't we can have a patched copy, but this would be a pain to maintain.
The problem is with YAML quoted string support in the library, if putting string in "…" you need to escape \, ", newline and carriage return character, but they forgot about escaping \.
This mode of encoding string is triggered by having a new line or carriage return character in string and then, any of the following character sequences would cause problems: \" (backslash will be omitted), \r (carriage return will be inserted instead), \n (new line will be inserted instead).
Theoretically all backslashes should be escaped in "…", but in case they are not followed by ", r or n they don't cause any problem.
When fixing the bug in library already saved strings with more than one backslash in row will get parsed (each \\ parsing to \), which also isn't expected.
I also found some YAML incompatibilities in sfYaml, but at least objects dumped and then parsed again is the same object (if the \ in multiline strings bug if fixed).
Piotr Gabryjeluk
visit my blog
So you're saying that even if the text was stored correctly, due to this YAML bug, it wouldn't decode properly anyway? Damn!
Why can't they implement your solution already? You've done the hard work for them!!!
Strings with newlines and backslashes were encoded wrongly (with backslashes not doubled), so they can be decoded as something else. But I hope this haven't affected many pages, because to encounter this bug, you need to:
Piotr Gabryjeluk
visit my blog
I used the API to create a page with "\\n" replacing all "\n"s. Whilst the source was stored correctly, it displayed wrong.
This is the source code of the page (stored correctly, because I forced it to via API):
This is what is displayed:
When I was expecting this:
And this is because not only dumper, but also parser is broken. If they don't sort this out till tomorrow, we're switching to our fork of the sfYaml library.
Piotr Gabryjeluk
visit my blog
We're using our fork of the library from now on. The issue should be fixed (with old sources working as explained above).
Piotr Gabryjeluk
visit my blog
Thanks Piotr!
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
Thank you!!!