ConlangDictionary 0.3 - now phonology parsing is faster

zelos · Post by **zelos** » Thu Apr 09, 2009 9:51 am

julianallees wrote:OH MY GOD faiuwle,

This would by far have to be the most best idea I have ever seen printed on the zompist forum. I cannot believe that I didn't think of even mentioning such a program, and the fact that you are creating it yourself is amazing. Are you going to include a sound button in your dictionary program in which you can hear a 'native' say the word ... correctly; because that would be amazing.

Anyway, well done, I cannot believe how brilliant your idea is, well done again.

Yours truly,

julianallees

he is going to add a word gen aswell, more specificly mine XD

HochDeutsch · Post by **HochDeutsch** » Mon Apr 13, 2009 6:59 pm

Ay.

I basically joined this forum to tell you how incredibly awesome this is, OP.

I've been looking for something like it for so long...
Bravo! Bravo!

This is a fantastic program that will revolutionize the way people (or at least I) conlang.
^_^

Linvi_Charles · Post by **Linvi_Charles** » Mon Apr 13, 2009 8:06 pm

When will the new version be available?

And, thank you for releasing this program!

faiuwle · Post by **faiuwle** » Mon Apr 13, 2009 9:00 pm

There was actually a bunch of people talking about getting together and doing something like this in Python. They made a wiki here, but I don't know if it ever got off the ground; I'm not very good at Python, and I'm not very good at working with a group, so I kind of bailed. But this definitely wasn't just my idea.

When will the new version be available?

Uh... I'm working on it. *shifty eyes* Some RL stuff came up.

faiuwle · Post by **faiuwle** » Thu Apr 30, 2009 2:00 pm

Fear not, ZBBians, I live and am actually still working on this. I know it's been a while; let me explain.

I originally thought the syllabification algorithm was going to be easy, but the whole process requires a lot more intuitive understanding of how syllables work, and Phonology 101 did not really hurt either. Basically, I had to rewrite it as a series of mind-fucking recursive functions, and after writing the first draft I had an urgent need to take an extended break from this whole thing. Fortunately, I have no returned to working on it, and the mind-fucking algorithms are working beautifully. There are still a few, mostly non-mind-fucking, things that have to be done before the next update:
- implement the last bits of the suprasegmental parsing
- GUI stuff
- savefile stuff

To tide you over until then, I bring you more shiny screenshots.

This is the new phonotactics tab. It is a little simplistic now, but it will probably improve over time. As you can see, adding 438273 clusters is actually pretty easy.

Here are some examples of the mind-fucking algorithms parsing words correctly, although my garden-variety (C)(C)V(C) phonotactics are hardly anything to complain about.

Linvi_Charles · Post by **Linvi_Charles** » Thu Apr 30, 2009 3:54 pm

This looks like it will be awesome. Thank you for making this program public!

Orthoptera · Post by **Orthoptera** » Fri May 01, 2009 12:27 pm

Wow, this is very cool! Thanks!

I'm having a bit of trouble with some IPA symbols though, they're only showing up as boxes (for example ɲ ɟ ɰ and ʎ)... I guess I'll just use XSAMPA for now.

zelos · Post by **zelos** » Fri May 01, 2009 2:10 pm

should also add export thing like lexique have

Jojo · Post by **Jojo** » Sat May 02, 2009 12:55 pm

This is awesome, I feel a better conlanger just because of this program.

A nice feature to have would be a phoneme selector (like in IPA table to choose glyphs from) as I can't switch the input method in the app forms to SCIM (GNOME user here, I suppose that to be the problem).
Also a sound change applier like the zompist's one would be really cool.

Nortaneous · Post by **Nortaneous** » Sat May 02, 2009 2:50 pm

Anyone compiled this for OSX yet?

faiuwle · Post by **faiuwle** » Sat May 02, 2009 9:39 pm

Orthoptera wrote:Wow, this is very cool! Thanks!

I'm having a bit of trouble with some IPA symbols though, they're only showing up as boxes (for example ɲ ɟ ɰ and ʎ)... I guess I'll just use XSAMPA for now.

That is odd; those characters do not work for me either in Windows, but they do show up in Linux. Qt claims to be able to display unicode characters in Windows even if they aren't supported natively... so I have no idea. Sorry.

Incidentally, a dialog in the next version will have an empty-set character - let me know if it shows up for you.

should also add export thing like lexique have

Yes, earlier people were talking about exporting to HTML and CSV, etc., which is all still going to happen eventually.

A nice feature to have would be a phoneme selector (like in IPA table to choose glyphs from) as I can't switch the input method in the app forms to SCIM (GNOME user here, I suppose that to be the problem).

If you just use the IPA characters for the names of the phonemes you technically only have to input them once - once you have defined your phoneme inventory you select phonemes via dialogs for various purposes.

Anyone compiled this for OSX yet?

Not that I know of, but you could volunteer if you're feeling brave.

zelos · Post by **zelos** » Sat May 02, 2009 11:04 pm

Yes, earlier people were talking about exporting to HTML and CSV, etc., which is all still going to happen eventually.

as in this centry or the next? =P i know i often use that phrase when i know it will take forever

Orthoptera · Post by **Orthoptera** » Thu Jun 25, 2009 10:09 am

Hey, any update on this?

LinguistCat · Post by **LinguistCat** » Thu Jun 25, 2009 5:48 pm

Yeah, keep us updated. I downloaded the first version, and it's a nifty little program. If I could just decide on phonotactics for a language...

dhok · Post by **dhok** » Thu Jun 25, 2009 6:46 pm

I hereby create the First Church of faiuwle. Thank you, O Divine Being, for making life easier for the pitiful likes of us.
Some suggestions:

-Put on for Windows what you've updated for Linux, and make sure we can update the software as needed without destroying the dictionary.
-Make an option so we can have as many sublevels as needed-one for gender, one for declension, one for article agreement, etc.
-Make an option for putting notes in, such as "This verb is not regular, as much as it may look at it, check the grammar before attempting conjugation."
-An option for diacritics would be nice.
-It would be very useful to be able to convert the dictionary to a text file.
-And, we have to create the texts, you know, they don't come ready to translate. The dictionary should be a two-way street.
-Not to pile a year's worth of work on you, but: what about a decliner/conjugator expansion pack connected to the dictionary? Enter the endings, take care of sandhi, and then when you enter any inflectable word in the language you get your paradigm. You could write an entire novel without getting up from your computer. And if we attached that to the Sound Change Applier, we'd have a truly revolutionary software.
Otherwise: Full steam ahead!

Qwynegold · Post by **Qwynegold** » Thu Jul 02, 2009 3:26 pm

I finally downloaded this to test it. This program might come in handy because I have a conlang with two different dialects, and it's hard to create a dictionary in LexiquePro that handles two languages equally. I can just create words in one dialect, save, load it, change the words that need to be changed and save that as dialect 2. But I have a worry, if I do it this way, I would have to create the whole dictionary in one sitting, because every time I click the load button I get the whole dictionary, including the words that I've already added. Is there a way solve this?

Also, I discovered a bug. I was trying to change the word kartoffel into perún. The acute accent is supposed to stand for half-long vowels. But I hadn't specified /ˑ/ or accented characters in the phonology tab, so when I clicked "change" with the box "automatically change pronunciation" box checked, the program just froze. Now it freezes every time I try to load that dictionary.

faiuwle · Post by **faiuwle** » Fri Jul 03, 2009 12:15 pm

dhokarena56 wrote:-Put on for Windows what you've updated for Linux, and make sure we can update the software as needed without destroying the dictionary.

This is already the case.

-Make an option so we can have as many sublevels as needed-one for gender, one for declension, one for article agreement, etc.
-Make an option for putting notes in, such as "This verb is not regular, as much as it may look at it, check the grammar before attempting conjugation."

It is all planned, don't worry.

-An option for diacritics would be nice.

It should already be possible to use diacritics.

-It would be very useful to be able to convert the dictionary to a text file.

It is saved as a text file. What specific formatting did you want?

-And, we have to create the texts, you know, they don't come ready to translate. The dictionary should be a two-way street.

You can already search in words or in definitions.

-Not to pile a year's worth of work on you, but: what about a decliner/conjugator expansion pack connected to the dictionary? Enter the endings, take care of sandhi, and then when you enter any inflectable word in the language you get your paradigm. You could write an entire novel without getting up from your computer. And if we attached that to the Sound Change Applier, we'd have a truly revolutionary software.

Yes, it is planned and will be awesome, but first I have to get the fucking savefile in the current version to work. Sigh.

Qwynegold wrote:I finally downloaded this to test it. This program might come in handy because I have a conlang with two different dialects, and it's hard to create a dictionary in LexiquePro that handles two languages equally. I can just create words in one dialect, save, load it, change the words that need to be changed and save that as dialect 2. But I have a worry, if I do it this way, I would have to create the whole dictionary in one sitting, because every time I click the load button I get the whole dictionary, including the words that I've already added. Is there a way solve this?

What do you want, here? An ability to just export the phonology plus a certain set of words to a new dictionary, or to open two dictionary files at once and transfer specific words between them? I want to eventually put some more configurable ways of linking languages/dialects diachronically, but like everything else, this is still in the future.

Also, I discovered a bug. I was trying to change the word kartoffel into perún. The acute accent is supposed to stand for half-long vowels. But I hadn't specified /ˑ/ or accented characters in the phonology tab, so when I clicked "change" with the box "automatically change pronunciation" box checked, the program just froze. Now it freezes every time I try to load that dictionary.

What does your savefile (.cdic) say about the kartoffel/perún word? Until I get the kinks in 0.2 ironed out, you can just specify ú as a spelling of /u/.

dhok · Post by **dhok** » Fri Jul 03, 2009 12:32 pm

faiuwle wrote:
dhokarena56 wrote:-Put on for Windows what you've updated for Linux, and make sure we can update the software as needed without destroying the dictionary.
This is already the case.

-Make an option so we can have as many sublevels as needed-one for gender, one for declension, one for article agreement, etc.
-Make an option for putting notes in, such as "This verb is not regular, as much as it may look at it, check the grammar before attempting conjugation."
It is all planned, don't worry.

-An option for diacritics would be nice.
It should already be possible to use diacritics.

Well, yes it is, but there's no "symbol" option-I have to copy and paste from Works, and dang is it slow.

-It would be very useful to be able to convert the dictionary to a text file.
It is saved as a text file. What specific formatting did you want?

Word-2003, please, and make sure Microsoft Works can take it.
[/quote]

-And, we have to create the texts, you know, they don't come ready to translate. The dictionary should be a two-way street.
You can already search in words or in definitions.

-Not to pile a year's worth of work on you, but: what about a decliner/conjugator expansion pack connected to the dictionary? Enter the endings, take care of sandhi, and then when you enter any inflectable word in the language you get your paradigm. You could write an entire novel without getting up from your computer. And if we attached that to the Sound Change Applier, we'd have a truly revolutionary software.
Yes, it is planned and will be awesome, but first I have to get the fucking savefile in the current version to work. Sigh.

Wonderful!

faiuwle · Post by **faiuwle** » Fri Jul 03, 2009 12:44 pm

dhokarena56 wrote:Well, yes it is, but there's no "symbol" option-I have to copy and paste from Works, and dang is it slow.

Do you have XP? If so, it's relatively easy to change your keyboard layout, at which point you can get most diacritics simply by typing ' + a for á, etc.

Word-2003, please, and make sure Microsoft Works can take it.

Can MS Works not open text files? Windows gets hung up on extensions, so you might have to rename it to .txt first. Also, Vim for Windows FTW.

Qwynegold · Post by **Qwynegold** » Wed Jul 29, 2009 2:22 pm

faiuwle wrote:
Qwynegold wrote:I finally downloaded this to test it. This program might come in handy because I have a conlang with two different dialects, and it's hard to create a dictionary in LexiquePro that handles two languages equally. I can just create words in one dialect, save, load it, change the words that need to be changed and save that as dialect 2. But I have a worry, if I do it this way, I would have to create the whole dictionary in one sitting, because every time I click the load button I get the whole dictionary, including the words that I've already added. Is there a way solve this?
What do you want, here? An ability to just export the phonology plus a certain set of words to a new dictionary, or to open two dictionary files at once and transfer specific words between them? I want to eventually put some more configurable ways of linking languages/dialects diachronically, but like everything else, this is still in the future.

I thought I had already replied to this, but I guess I hadn't. Anyway, if each word had a field that said what time and date it was added, and the program automatically filled that in every time you created a word, then it would be really great if you could specify to load only words that were created after a certain date. Would this be difficult to implement? How is the programming going btw, are you still working with it?

Pogostick Man · Post by **Pogostick Man** » Mon Aug 03, 2009 8:16 pm

When I try to use it, it keeps giving me the following error:

"This application has failed to start because QTCore4.dll was not found. Re-installing the application may fix this problem."

How do I go about rectifying this?

faiuwle · Post by **faiuwle** » Tue Aug 04, 2009 4:23 am

Qwynegold wrote:I thought I had already replied to this, but I guess I hadn't. Anyway, if each word had a field that said what time and date it was added, and the program automatically filled that in every time you created a word, then it would be really great if you could specify to load only words that were created after a certain date. Would this be difficult to implement?

Adding timestamps to word entries wouldn't be a problem, but I don't think it would really work. I'm guessing you don't want to load the whole dictionary because you have a long wordlist and it takes a while to load? The thing is, in order to save an updated version of the file, it will have to read the old file anyway (all of it), which means loading that data into memory so that the program can make sense of it.

Or (option 2) do you want something more like a separate file for the second dialect, and after opening that file use the program to scan the first file for new vocabulary entries? That might work, especially if I wind up revamping the savefile format again soon, which I think I may do.

Option 3 would be to have the program load multiple dictionaries simultaneously, but that could get out of hand for really big files, and besides, it feels bulky and inelegant.

Option 4 might be to define multiple dialects in the same file, maybe with two sets of phonotactics. Individual word entries might then have a field for the form in the second dialect (if different) but would still only have one comment/POS/etc. field as usual. Of course, that means that words might then have an extra phonological representation for every additional dialect, but I don't think it would be that much trouble. I guess the only way to find out if it is is to try it.

How is the programming going btw, are you still working with it?

Yeah, sorry for the whole vaporware thing. I was running into savefile-related issues last time I put it aside, and managed to overwrite a recent dictionary of mine with a broken savefile, which was kind of frustrating. (I have only myself to blame; I really need to stop trying to actually use programs that are still in an incomplete state.) Anyway, I've also been in the process of moving and setting up a new place, and finishing applying to a new school, etc., etc., and haven't really been thinking about it. However, I now find myself pretty much done with all of that (well, less one trip to Ikea), and I've still got a month of nothing much to do left, so I'm going to try and rewrite the save function, possibly into something more systematized like honest-to-god XML or something. That would save me having to make an XML output file too, I guess.

[/mememe]

I'll try to get the savefile working by the end of the month - really.

Rorschach wrote:When I try to use it, it keeps giving me the following error:

"This application has failed to start because QTCore4.dll was not found. Re-installing the application may fix this problem."

How do I go about rectifying this?

You need the Qt Libraries for Windows. Unzip that file in the same directory as the program's executable.

Qwynegold · Post by **Qwynegold** » Tue Aug 04, 2009 11:33 am

faiuwle wrote:
Qwynegold wrote:I thought I had already replied to this, but I guess I hadn't. Anyway, if each word had a field that said what time and date it was added, and the program automatically filled that in every time you created a word, then it would be really great if you could specify to load only words that were created after a certain date. Would this be difficult to implement?
Adding timestamps to word entries wouldn't be a problem, but I don't think it would really work. I'm guessing you don't want to load the whole dictionary because you have a long wordlist and it takes a while to load? The thing is, in order to save an updated version of the file, it will have to read the old file anyway (all of it), which means loading that data into memory so that the program can make sense of it.

Or (option 2) do you want something more like a separate file for the second dialect, and after opening that file use the program to scan the first file for new vocabulary entries? That might work, especially if I wind up revamping the savefile format again soon, which I think I may do.

Option 3 would be to have the program load multiple dictionaries simultaneously, but that could get out of hand for really big files, and besides, it feels bulky and inelegant.

Option 4 might be to define multiple dialects in the same file, maybe with two sets of phonotactics. Individual word entries might then have a field for the form in the second dialect (if different) but would still only have one comment/POS/etc. field as usual. Of course, that means that words might then have an extra phonological representation for every additional dialect, but I don't think it would be that much trouble. I guess the only way to find out if it is is to try it.

Actually, option #2 would be best for what I need now.

faiuwle wrote:
How is the programming going btw, are you still working with it?
Yeah, sorry for the whole vaporware thing. I was running into savefile-related issues last time I put it aside, and managed to overwrite a recent dictionary of mine with a broken savefile, which was kind of frustrating. (I have only myself to blame; I really need to stop trying to actually use programs that are still in an incomplete state.) Anyway, I've also been in the process of moving and setting up a new place, and finishing applying to a new school, etc., etc., and haven't really been thinking about it. However, I now find myself pretty much done with all of that (well, less one trip to Ikea), and I've still got a month of nothing much to do left, so I'm going to try and rewrite the save function, possibly into something more systematized like honest-to-god XML or something. That would save me having to make an XML output file too, I guess.

[/mememe]

I'll try to get the savefile working by the end of the month - really.

I hope I haven't pressured you too much.

But great that you're gonna work on it again now.

faiuwle · Post by **faiuwle** » Tue Aug 04, 2009 7:15 pm

No problems - sometimes I do need someone to bother me about something in order to actually get it done.

Actually, option #2 would be best for what I need now.

Actually, looking over the Qt DOM XML stuff, it seems like it would be pretty easy to do, and since it doesn't involve changing the savefile (the timestamp can be incorporated into the savefile format I'm working on now) it can be part of 0.2. This would just be importing the spelling, though, and the current dictionary would recalculate the phonology based on the spelling, or I could put in an option to set it up manually on import.

faiuwle · Post by **faiuwle** » Wed Aug 05, 2009 2:41 am

Yeah, Qt's XML classes are going to be easy as pie. Awesomeness.

Code: Select all

<!DOCTYPE ConlangML>
<conlang name="ZOMGConlang" >
 <inventory>
  <phonemeDef name="a" >
   <orthography>a ah aa</orthography>
   <notes>Some generic low vowel.</notes>
  </phonemeDef>
  <phonemeDef name="e" >
   <orthography>e eh ei</orthography>
   <notes>/e/ is boring.</notes>
  </phonemeDef>
  <phonemeDef name="i" >
   <orthography>i ie y ee</orthography>
   <notes>/i/ can has Englishish orthography!</notes>
  </phonemeDef>
  <phonemeDef name="o" >
   <orthography>o oh ou ow</orthography>
   <notes>Ow!  Oh.</notes>
  </phonemeDef>
  <phonemeDef name="u" >
   <orthography>ú uu uw</orthography>
   <notes>Are diacritics preserved?</notes>
  </phonemeDef>
  <phonemeDef name="y" >
   <orthography>y iw ew ü</orthography>
   <notes>Front rounded vowels are so 1337.</notes>
  </phonemeDef>
  <phonemeDef name="p" >
   <orthography>p ph pp</orthography>
   <notes>Running out of ideas here.</notes>
  </phonemeDef>
  <phonemeDef name="t" >
   <orthography>t tt t'</orthography>
   <notes>Yawn.</notes>
  </phonemeDef>
  <phonemeDef name="k" >
   <orthography>k qu ck</orthography>
   <notes>The quick brown fox...</notes>
  </phonemeDef>
  <phonemeDef name="b" >
   <orthography>d</orthography>
   <notes>Yes, /b/ is randomly spelled <d>.  (Look, angle brackets!  Can you hack it, XML?)</notes>
  </phonemeDef>
  <phonemeDef name="d" >
   <orthography>b</orthography>
   <notes>Well, what did you expect?</notes>
  </phonemeDef>
  <phonemeDef name="g" >
   <orthography>g gg gu gh</orthography>
   <notes>It's /g/.  What more do you want?</notes>
  </phonemeDef>
 </inventory>
</conlang>