Page Bottom  References from Other Sources

Integrated Databases

Esther Lyman says, I agree with Pieter and functionally we DO have a standard ... comma-delimited text files; which will work fine as long as we can keep the national database clean of extraneous COMMAs. We'll work to that end. Most of the vendors requesting mailing lists ask for comma-delimited files.

I have no problem with going with Access and those who wish can do it because most DB files will import into it. In fact, I do use it to "rescue" files done in some of Leo's old software (and which turn out to be DB3 files) by simply importing them into Access and exporting them again into whatever format I need. But I find the strict validation rules to hamstring me in getting work done because of the myriad of systems the data originates on. This very fact, of course, makes us wish for standization, but I don't see how this is workable right now.

Please consider also that the National Office is currently using Excel as a working environment because we are tasked with several other responsibilities other than the downstream databases. We need to have tools to run comparisons to weed out duplicate mailing addresses, duplicate registrations, incorrect club numbers / abbreviations and more. With Excel we can easily do all this right off the database. We need to check club registrations against clubs registered and more.

We also have to cover the accounting piece, and, again, we can run reports right off the database so I can compare what should be received with what Tracy did receive, and report back to the clubs.

I would say that we could easily output Access files for those who wish, but the fact is that it is not easy. Because the data originates from so many levels of systems and software, so much of the data gets rejected in important that we would spend too much time cleaning it up now. Possibly when most registrars come up on Leo's newest version, we'll have more consistent data. But we have no trouble outputting generic .DBF files for you.

So, pleease bear with us while we figure out how to best serve all masters, while reducing the overlap in work for the registrars, us, and downstream users. We've already eliminated a lot of extra work/grief for everybody. Let's revisit this when we see how the last version of the 99 database comes out and what you can do with it.

Best, Es-

Eric Baelen says: Interesting note. But I think you are fighting yesterdays war. The idea of standardizing upon a database has been overtaken by XML. If an XML set of tags and formats were to be agreed upon, all of the modern databases (including Oracle and Access) can use it. Internet communications about events could be immediately sent using it and automatically decoded and updated. Support for ODBC and/or ADO would also be of interest. /Eric Baelen
I very much share your concern about having back-ups, not only for data but also for people. One of the terms in this equation is a data-interchange format that can be used across platforms.

You realize that we have a diversity of computer talent using all kinds of software packages. It therefore would be prudent to agree on a data interchange format that is easy to use and "universal" That way anyone that wishes to participate can do so using their own preferred software environment. This implies that we need to stay away from proprietary formats like Excel and Access and even dBase. There is no guarantee that our friends at Microsoft will not make changes to the formats that makes them incompatible.

As you know I have always been an advocate of using comma-delimited text formats.

This format can very easily be imported into Access, Excel and dBase, which are probably the most widely used programs.

You may consider it a disadvantage that the field names are often not included in the text file. On the other hand, not including field names has the advantage that we do not have to coordinate field names across different applications. Only the order of the fields within each record is of some importance. Once you have the data in e.g. Excel, you can easily move the columns. Often field names are obvious from the field content.

Another advantaged is that field length does not have to be standardized. This format is also very easy to export from these applications.

Pieter Cath.

This e-mail is about my approach to computers and why it is useful for us to have other computer talents on our commiteee.

I work in a language most people haven't heard of and people who have heard of it don't like it. It is a very powerful language, and it has been around for a very long time (since 1962). It is called "APL". I believe that anybody skilled in APL can do complicated work far more rapidly than anybody working in any other language. But people don't learn it because they think it is complicated and has too much math; believe it or not many computer people do not like math. I am doing my best to figure out how to keep data in a format that will be available to lots of other people, but it is not easy because I only have one of those other people to easily and happily exchange with at a technical level (Esther Lyman). (This week I sent her an Excel file of all the permanent swimmer id's I've assigned and she reports that it is a high quality Excel file. I am very happy to hear that.)

I also think Esther is like me -- a problem solver -- not a traditional systems person.

It may take a great deal more effort to be a traditional systems person. They are concerned with documentation, and how can we see that what we have will outlive whoever is working on the project at the moment. Leo's work is probably the closest we have to traditonal systems work going on in USMS right now. (If others disagree, please understand that this is just my point of view and possibly the beginning of a dialogue.)

I believe we should develop preferred computer formats into which we should put data whenever we are able. Before convention I thought it should be in Oracle, the most widely used database language in Fortune 500 companies. But Jim Matysek and Paul Wright convinced me it should be in Access which is far easier to use and far more widely used in small organizations. So, it's Access. But there is more to do than simply choose the language. We need to agree on field parameters and field combinations for various uses. And I'm sure the skilled Access person can describe many more things we need to agree on or make easy. I think there is an ethic in USMS (and maybe in lots of computer communities) where pride is taken in being able to get a job done. I submit that is sometimes a shortsighted pride. The greatest pride should be taken in tasks that move the community ahead and that make all our tasks a little bit easier.

Excel is certainly not appropriate as our preferred format. It is like my approach, very powerful but not at all appropriate for sophisticated databases. (My approach is, by the way, appropriate for sophisticated databases, but only if there are other people who can use them and carry on if the creator is not available. I don't think that is the case in USMS.)

There needs to be 1 or 2 or 3 people who are skilled Access people who will help all of us design how all the information we are gathering will be captured for future generations and for use by people other than me. I think these must be people who are not already overburdened with USMS responsibilities.

I'll be very happy when we find those people I can collaborate with. There is a need for one or two of them to be on the History and Archives Committee.

horizontal line
to home page e-mail Page Top