How do you store your lexicon?

Museum for the best conlanging and conworldery threads. Ask mods to move threads here.
zoqaëski
Sanci
Sanci
Posts: 28
Joined: Tue Apr 05, 2005 7:01 am
Location: Hiding

How do you store your lexicon?

Post by zoqaëski »

I'm pretty sure a number of threads on this topic have sprung up recently, but searching couldn't find them so my apologies for creating a duplicate thread if I have done so.

How do you store your conlang's lexicon?

I'm up to the stage with my Qevesa grammar that I need to start building up a lexicon so I can participate in TCs to further refine the grammar. Problem is, I've got no idea how to store it, or even where to start; I've experimented with Text Documents or CSVs in OpenOffice, XHTML definition lists, plain text files, and none of them are satisfactory. Faiuwle's ConlangDictionary looks interesting but I'm not sure if it would work well with what I have in mind.

Ideally I'd like to create a searchable database accessible from my browser, using a SQLite or similar backend, that's simple to maintain and extensible. I quite like Guitarplayer's dictionary for Ayeri on his TayBenung site, and there's a few others I've seen that look like a well-designed interface (although I have no idea how well they work internally).

The main problem is that as Qevesa is based around consonantal roots, so I really need a means of storing all the derivative forms, and searching them all, as well as being able to edit them with ease. The resulting system could be visualised as a 3D table (X=vowel patterns / Y=roots / Z=meanings), so a database would probably be ideal, but trying to develop this is a seemingly insurmountable task. Where should I start?
Zoqaëski : /θoˈca.jes.ki/
Last.fm
Twitter

User avatar
Jipí
Smeric
Smeric
Posts: 1128
Joined: Sat Apr 12, 2003 1:48 pm
Location: Litareng, Keynami
Contact:

Re: How do you store your lexicon?

Post by Jipí »

Aw do we have to have that thread A G A I N ? (Sorry!) What you may want to have a look at is SQL syntax, Codd's 12 Rules, and how inner joins work. Also, how to make queries and fetch them with e.g. PHP. Yes, it requires some work, but it's not unworkable.

User avatar
Aurora Rossa
Smeric
Smeric
Posts: 1138
Joined: Mon Aug 11, 2003 11:46 am
Location: The vendée of America
Contact:

Re: How do you store your lexicon?

Post by Aurora Rossa »

I use a Microsoft Word file with lists of lexemes, place names, personal names, and so forth.
Image
"There was a particular car I soon came to think of as distinctly St. Louis-ish: a gigantic white S.U.V. with a W. bumper sticker on it for George W. Bush."

bulbaquil
Lebom
Lebom
Posts: 242
Joined: Fri Nov 17, 2006 2:31 pm

Re: How do you store your lexicon?

Post by bulbaquil »

Excel.
MI DRALAS, KHARULE MEVO STANI?!

User avatar
Risla
Avisaru
Avisaru
Posts: 800
Joined: Sun Mar 25, 2007 12:17 pm
Location: The darkest corner of your mind...

Re: How do you store your lexicon?

Post by Risla »

Google Docs spreadsheet. I've also got a notebook where I write down new words, but the spreadsheet is much easier to organize.

Acid Badger
Lebom
Lebom
Posts: 196
Joined: Tue May 11, 2010 5:50 pm
Location: Berlin, Germany

Re: How do you store your lexicon?

Post by Acid Badger »

Excel.

User avatar
WeepingElf
Smeric
Smeric
Posts: 1630
Joined: Wed Mar 08, 2006 5:00 pm
Location: Braunschweig, Germany
Contact:

Re: How do you store your lexicon?

Post by WeepingElf »

A simple but well-formatted HTML file, in which each lexeme sits on a line for itself.
...brought to you by the Weeping Elf
Tha cvastam émi cvastam santham amal phelsa. -- Friedrich Schiller
ESTAR-3SG:P human-OBJ only human-OBJ true-OBJ REL-LOC play-3SG:A

User avatar
makvas
Avisaru
Avisaru
Posts: 251
Joined: Wed Jul 19, 2006 6:13 pm
Location: The Southland

Re: How do you store your lexicon?

Post by makvas »

Google Docs spreadsheet, since it's a collaborative project. You can have multiple pages in one spreadsheet file, which is handy for organization.

User avatar
personak
Sanci
Sanci
Posts: 25
Joined: Thu Oct 14, 2010 12:31 pm
Location: USA
Contact:

Re: How do you store your lexicon?

Post by personak »

Lexique Pro is great.
Remember, kids. Aspirate your initial plosives!
http://nathansoftware.blogspot.com

FOSCA

Bedelato
Lebom
Lebom
Posts: 193
Joined: Sat Oct 30, 2010 1:13 pm
Location: Another place

Re: How do you store your lexicon?

Post by Bedelato »

For a long time I used Excel, but I'm currently in the process of migrating everything to Lexique Pro.
At, casteda dus des ometh coisen at tusta o diédem thum čisbugan. Ai, thiosa če sane búem mos sil, ne?
Also, I broke all your metal ropes and used them to feed the cheeseburgers. Yes, today just keeps getting better, doesn't it?

Bristel
Smeric
Smeric
Posts: 1258
Joined: Mon Jun 01, 2009 3:07 pm
Location: Miracle, Inc. Headquarters
Contact:

Re: How do you store your lexicon?

Post by Bristel »

Pages or Numbers.
[bɹ̠ˤʷɪs.təɫ]
Nōn quālibet inīquā cupiditāte illectus hoc agō
Yo te pongo en tu lugar...
Taisc mach Daró

User avatar
Ashroot
Lebom
Lebom
Posts: 99
Joined: Fri Jan 21, 2011 12:24 am

Re: How do you store your lexicon?

Post by Ashroot »

I use to use 3x5 cards for the few words I had but now I am going to Lexique Pro.

User avatar
Foolster41
Lebom
Lebom
Posts: 129
Joined: Wed Oct 28, 2009 5:55 pm
Location: Pacific Northwest
Contact:

Re: How do you store your lexicon?

Post by Foolster41 »

I have a word doc, and a dictonary on my wiki.

WanderlustKoko
Lebom
Lebom
Posts: 112
Joined: Sat Aug 02, 2008 2:23 pm
Location: Fast i mitt eget huvud.

Re: How do you store your lexicon?

Post by WanderlustKoko »

Open Office at the moment. I believe the have a similar thing to excel in it as well.
Current Conlang Project: Karuslnko. http://karuslnkoconlang.wordpress.com

User avatar
alice
Avisaru
Avisaru
Posts: 707
Joined: Wed Oct 30, 2002 4:43 pm
Location: Three of them

Re: How do you store your lexicon?

Post by alice »

In smallish glass jars. I find almond vinegar preserves it best.
Zompist's Markov generator wrote:it was labelled" orange marmalade," but that is unutterably hideous.

User avatar
Chuma
Avisaru
Avisaru
Posts: 387
Joined: Sat Oct 28, 2006 9:01 pm
Location: Hyperborea

Re: How do you store your lexicon?

Post by Chuma »

As for me, I use a Numbers file. I don't need anything more complicated.

I took a course in SQL a while back, but I didn't find it particularly useful. I don't think you would need anything like that for your problem.

I might be able to help you, if you can a) explain exactly what you need, and b) run Perl programs.

I'm not sure why you would need a rank 3 array. I would have guessed roots on one axis, patterns on one axis, and for each entry the English word (and the conlang word, if that's not completely obvious from root+pattern). Then you could search for either an English word, a conlang word, or a root+pattern, and see all matching entries, or you could search for a root, and see a list of all patterns which are possible with that root (and perhaps their whole entries).
Is that what you need, or are there any other searches you need to be able to do?

User avatar
Jipí
Smeric
Smeric
Posts: 1128
Joined: Sat Apr 12, 2003 1:48 pm
Location: Litareng, Keynami
Contact:

Re: How do you store your lexicon?

Post by Jipí »

Chuma, I think you're suggesting basically what a Join does, except you'd deal with dynamically generated results instead of a static list:

Code: Select all

SELECT root, pattern, translation
FROM tbl_root, tbl_pattern, tbl_translation
WHERE root LIKE "x-y-z"
   AND tbl_root.root_id = tbl_pattern.root_id
   AND tbl_root.rood_id = tbl_translation.root_id;
Something like this. At least that's kind of what I'm doing for my database. And for the purpose of a dictionary IMNSHO a properly set-up relational database is not useless or overkill. And you can generate derivative lists in different formats easily.

User avatar
Z500
Lebom
Lebom
Posts: 85
Joined: Sun Jan 30, 2005 3:01 pm
Contact:

Re: How do you store your lexicon?

Post by Z500 »

in individual text files for easy feeding to IPA Zounds
scientists have discovered a capsule that makes you not a gullible fucktard!

zoqaëski
Sanci
Sanci
Posts: 28
Joined: Tue Apr 05, 2005 7:01 am
Location: Hiding

Re: How do you store your lexicon?

Post by zoqaëski »

Chuma wrote:I'm not sure why you would need a rank 3 array. I would have guessed roots on one axis, patterns on one axis, and for each entry the English word (and the conlang word, if that's not completely obvious from root+pattern). Then you could search for either an English word, a conlang word, or a root+pattern, and see all matching entries, or you could search for a root, and see a list of all patterns which are possible with that root (and perhaps their whole entries).
Is that what you need, or are there any other searches you need to be able to do?
That's exactly what I'm looking to do, now that I've got an unlimited net connection I'm going to start reading up on SQLite so I don't have the overhead of something as big as mySQL or whatever. Basically, the table keys would be root (c1-c2-c3) × pattern (v0-v1-v2-v3), and the translation and information in the matrix cells (the 3rd dimension could list fields such as conlang word, pronunciation, English meaning(s), etc), although the actual conlang word could be generated on the fly from some basic regex, I guess.
Guitarplayer wrote:Chuma, I think you're suggesting basically what a Join does, except you'd deal with dynamically generated results instead of a static list:

Code: Select all

SELECT root, pattern, translation
FROM tbl_root, tbl_pattern, tbl_translation
WHERE root LIKE "x-y-z"
   AND tbl_root.root_id = tbl_pattern.root_id
   AND tbl_root.rood_id = tbl_translation.root_id;
Something like this. At least that's kind of what I'm doing for my database. And for the purpose of a dictionary IMNSHO a properly set-up relational database is not useless or overkill. And you can generate derivative lists in different formats easily.
That's the idea. Presumably I could use a SQLite DB for the backend, and generate static lists as I need for export, extracting the useful information I want.

The trick is to keep this whole scheme simple (i.e. KISS principle) but extensible and easy-to-maintain.
Zoqaëski : /θoˈca.jes.ki/
Last.fm
Twitter

User avatar
masako
Smeric
Smeric
Posts: 1731
Joined: Sat Nov 06, 2004 4:31 pm
Location: 가매
Contact:

Re: How do you store your lexicon?

Post by masako »

Image

User avatar
Chuma
Avisaru
Avisaru
Posts: 387
Joined: Sat Oct 28, 2006 9:01 pm
Location: Hyperborea

Re: How do you store your lexicon?

Post by Chuma »

Zoqaeski wrote:the 3rd dimension could list fields such as conlang word, pronunciation, English meaning(s), etc
Oh, I see. Yeah, that could be thought of as a third dimension in a table, depending on the application.
Guitarplayer wrote:Something like this.
Looks sensible.

Phew, I feel like I've forgot all about SQL. Let's see... what tables might we have?

tbl_root (root_id, root)
tbl_pattern (pattern_id, pattern)
tbl_translation (translation_id, translation, root_id, pattern_id)

Then you should be able to do

Code: Select all

SELECT root, pattern, translation
FROM tbl_root NATURAL JOIN tbl_pattern NATURAL JOIN tbl_translation
WHERE root = "x-y-z"
I'm not sure I understand your way of doing it - why does tbl_pattern have a root_id?

Anyway, that should work, but it doesn't sound like the simplest way to me. Not sure how clever the SQL interpreter is, but if you do a straight join it would have a huge list to look through, so that's hardly effective (not that it should matter unless you have a really huge lexicon).

Basically all you need is one ordinary table. I also assume that you have a finite constant number of patterns, which makes things easier if you store the table in a file. You just make a list, where each line has the different patterns for a single root. That seems much easier to me than making an SQL database just for this one wordlist. But hey, do as you like.

User avatar
justin
Sanci
Sanci
Posts: 52
Joined: Tue Mar 23, 2010 5:17 pm
Contact:

Re: How do you store your lexicon?

Post by justin »

personak wrote:Lexique Pro is great.
You wouldn't happen to know how to get Lexique Pro to treat capital letters the same as lowercase when it sorts, would you? I have proper nouns in my wordlist, and they sort before the ones that aren't capitalized.
Soî yelî sanoralî er verdî dormü gurišece.
Se vŕeȥe ili buz orarn dŕmn gulregi.
Economic Left/Right: -5.62
Social Libertarian/Authoritarian: -4.92

zoqaëski
Sanci
Sanci
Posts: 28
Joined: Tue Apr 05, 2005 7:01 am
Location: Hiding

Re: How do you store your lexicon?

Post by zoqaëski »

Chuma wrote: Phew, I feel like I've forgot all about SQL. Let's see... what tables might we have?

tbl_root (root_id, root)
tbl_pattern (pattern_id, pattern)
tbl_translation (translation_id, translation, root_id, pattern_id)

Then you should be able to do

Code: Select all

SELECT root, pattern, translation
FROM tbl_root NATURAL JOIN tbl_pattern NATURAL JOIN tbl_translation
WHERE root = "x-y-z"
I'm not sure I understand your way of doing it - why does tbl_pattern have a root_id?

Anyway, that should work, but it doesn't sound like the simplest way to me. Not sure how clever the SQL interpreter is, but if you do a straight join it would have a huge list to look through, so that's hardly effective (not that it should matter unless you have a really huge lexicon).

Basically all you need is one ordinary table. I also assume that you have a finite constant number of patterns, which makes things easier if you store the table in a file. You just make a list, where each line has the different patterns for a single root. That seems much easier to me than making an SQL database just for this one wordlist. But hey, do as you like.
Both biliteral and triliteral roots are permissible, and there are 22 consonant phonemes. So that makes 22^2 + 22^3 = 11132 possible roots... not that I think I'll use all of them, of course. I'm only going to store nominal and adjectival forms and the verbal infinitive, because that will keep the number of patterns down to a manageable level. Besides, there's no point in having a dictionary entry for every aspectual conjugation, is there?

What I don't really understand is whether I can actually store the database as a table reminiscent of an addition or multiplication table, because logically root + pattern = word. If I give IDs to every root and pattern, the cells could be identified by some kind of mathematical relationship, and each cell could then consist of a "document" containing the necessary information fields.

EDIT: Reading some of the SQLite (and SQL in general) documentation, it seems that the best idea will be to have separate tables for roots and patterns, and searching performs a join. The earlier idea of having a table with a column for each pattern might be impossible to implement, because the actual table will need to resize, and I don't think that's possible.
Zoqaëski : /θoˈca.jes.ki/
Last.fm
Twitter

Bedelato
Lebom
Lebom
Posts: 193
Joined: Sat Oct 30, 2010 1:13 pm
Location: Another place

Re: How do you store your lexicon?

Post by Bedelato »

sano wrote:Image
What the...

EXCEL 97 !!? :o :o :o :o :o
At, casteda dus des ometh coisen at tusta o diédem thum čisbugan. Ai, thiosa če sane búem mos sil, ne?
Also, I broke all your metal ropes and used them to feed the cheeseburgers. Yes, today just keeps getting better, doesn't it?

User avatar
Chuma
Avisaru
Avisaru
Posts: 387
Joined: Sat Oct 28, 2006 9:01 pm
Location: Hyperborea

Re: How do you store your lexicon?

Post by Chuma »

Zoqaeski wrote:What I don't really understand is whether I can actually store the database as a table reminiscent of an addition or multiplication table, because logically root + pattern = word. If I give IDs to every root and pattern, the cells could be identified by some kind of mathematical relationship, and each cell could then consist of a "document" containing the necessary information fields.
That's pretty much it, yeah. In some languages you would need numerical IDs to identify the roots, in others you could use the root itself as key. But that's all pretty basic programming, and won't affect the finished program.
Zoqaeski wrote:Reading some of the SQLite (and SQL in general) documentation, it seems that the best idea will be to have separate tables for roots and patterns, and searching performs a join.
Yes, that would be the normal way to do things in SQL, I think. Not in other programs.

Post Reply