Helmut,
The key is that the content of both the structured and unstructured data must be on the same page otherwise this won't work at all. The structured content describes or in some way identifies the unstructured content to make it sortable in ways that cannot be done now, at least not without a lot of manual effort. For example, if I wanted to associate the name of a page's author with the page, I could create an entire data form category that contained just author names and links to pages, but that would be a lot of extra work for the people creating the content on my site to copy the URL of their page, create a new page in this author's category and paste the link in the new page and enter their name. It would be much easier if there were a text box or select box where they could enter their name right on the page they are writing so that their name will be directly associated with the page containing their content.

This way I could use ListPages to sort pages by author (not by page creator) or any other criteria I define in the data form portion of the page.

This is just one example of how it could be used. The flexibility of this system would unleash a whole series of applications that are not currently possible.

That is almost exactly what I'm trying to do. However, when I tried something like this, includes didn't work in a wiki field. They didn't get parsed at all. I was told in the forums that the wiki field doesn't support includes which are used extensively by my site as a way of extending the wikidot markup. Apparently includes do work according to your page, so I shouldn't give up on data forms just yet.
However, beyond that there are major differences in my proposal. If data forms and unstructured content could be combined:

• I could much more easily retrofit an existing wiki to use data forms simply by adjusting the current live template to include a data form portion.
• Creating and editing the wiki content portion would be no different than it is now, with the familiar editor box and its tool bar.
• Displaying the data entered on a form could be achieved more easily since you would simply be adding variables in a familiar unstructured wiki page. It wouldn't be necessary for an end user to use CSS and jump through hoops just to get the fields to display the way they want.
• I could use static templates with data form variables in conjunction with a combined data form/live template to provide several different ways of presenting a single page that an author could choose when creating the page.

Thanks for your example, Timothy. This certainly illustrates some things that aren't clear from the data form documentation. I've really struggled with data forms since the documentation is incomplete or scattered around in separate areas of Wikidot. This gives me hope that I can do what I want even if it will be fairly labor intensive to get there.

This is a very creative solution given the current implementation of data forms. However, it seems like it is creating an extra burden on the servers since it uses ListPages to get and display the current page's content. I would think that my proposal would ultimately be more efficient since it only requires the markup parser to pick up what is on the current page and place the data, also on the current page, in the appropriate places when the page is compiled/rendered. Of course I could be wrong since I don't know exactly how either ListPages or the parser/compiler are implemented.

Timothy, I've played around with your solution a bit, and it almost works for me. Where it fails is that I need to be able to include wiki fields from one data form inside the wiki field of another data form, just as I would include one regular page's content inside another page's content. That doesn't appear to be possible when the included page is itself a data form. Using a regular include just lists all the content of the data form listed by field. Using ListPages inside the wiki field doesn't work since ListPages can't be used recursively.

As it turns out, my proposed solution probably wouldn't be any better since includes in effect copy the content of one page to the other and then render it. So any references to the first page's variables would no longer be valid on the second page where the content was included. What would be needed is something that behaves more like an iframe where the included content is rendered independently of or before being included on the host page.

You can include a dataform page with a ListPages module:

[[module ListPages category="categoryname" name="pagename" limit="1"]]
whatever you want
[[/module]]

However, this will not work in the example I gave you because it relies on a ListPages module for self-reference. Modules cannot go inside each other, so it would fail.

You can amend that, however, by not having the self-referential ListPages module on the live template page and instead use it in the content fields of each page where it is needed.

Thanks for the suggestion, but I'm not sure that gets me to where I need to be.

Both the source page and the host page where the ListPages would reside in the content field must be viewable as if they were ordinary wiki pages.

If I eliminate the ListPages from the template, the source page can only be displayed as a list of fields.

I appreciate the discussion. I've learned a lot about data forms that will be useful for some other things I'm working on, but I don't think I can use them for this particular part of my project. They come tantalizingly close to meeting my needs, but fall just a little short. All I really want is to be able to attach a few data fields to what is otherwise a completely normal wiki page. In many respects, what I need is a much more flexible and robust tagging system than we have now.

Then I could finally sort pages by author, which is impossible to do now.

No it's not. If you have a text fieldtype called author you can sort by the author in the ListPages module using order="_author" or order="_author desc" (make sure you use the underscore) .

You've not said what is missing from the documentation but sorting by data form field is certainly there.

I can sort by author IF I'm using a data form. I meant that with a regular wiki page it is not possible. That's why I'd like to be able to combine data forms and regular wiki content in a more flexible way.

Keep in mind, with a regular wiki page the author isn't necessarily the page's creator. The wiki engine has no idea who the author of the content is without looking at the history, which list pages can't really do.

I finally found %%form_raw{name}%% on the ListPages documentation page. It's description isn't very helpful for this application, "For select fields, the internal value saved in the page form data, if any". Nowhere does it mention being useful for wiki fields (it suggests it is only for select fields) unless you happen to see a comment in a code box that demonstrates "Ordering Pages" half way back to the top of the page:

[!--
Note: Use %%form_raw{fieldname}%% for wiki field types,
         %%form_data{fieldname}%% for other field types
--]

form_raw is also discussed in some detail in relation to images at http://www.wikidot.com/doc-data-forms:images although I don't think you are intending to use it with images.

I don't doubt that it might take a little extra work to parse two different languages on a page, but if they are separated it would be relatively trivial to call one parser then the other. For example, my request could be simplified to have the data form content always be at the bottom. Then, the page parser could go through the page like it does normally. When it hits the separator it switches to the YAML parser to pick up the data form portion. I've done things like this myself, so I know it's possible. Web browsers do this all the time when they switch from HTML to XML (used by SVG images). Now, there may be other complicating factors I'm not aware of here, so I would understand if something else made it non-trivial.

It's times like these that I really wish WikiDot were still open source.

I really appreciate the discussion with you and Timothy. I've learned a few things about data forms I didn't know before that will prove useful even if I don't get to use them for what I'm working on now.