Tips for Conversion ________________
 
Aim: 
Conversion should be performed in as few manual steps as is possible. 

Problem Definition: 
To convert between any given scheduler and pilot datebook database format. 
 
Tools: 
CONVDB, any other support programs: perl, gawk (gnu awk), ksh, bash.... basically I am listing all of unix tools. Now, in win95/NT this is the roadblock -- these tools are not intrinsic to these operating systems. Yes, you can download them from various sources, but not many people have the time or inclination or the stomach to install these tools. Solution: Tough! the only other alternative I see is to write a customized tool and compile. To me that is not an alternative. It takes me a few minutes to write script for one of the unix tools to do the job. 
 
Win95/NT batch files and commands amount to a speck against unix tools. So, the best thing to do is to download a distribution. The best source today is Cygnus tools, download the usertools.exe which contains a complete package with instructions to install it, if you plan to do any kind of data conversion you will not regret it. 
If you think you would rather just get individual tools as you go, then search at  shareware.com under the "all categories". 
Cygnus :  GNU-Win32 Project Page 
List of tools I would definitely have: 

  1. ksh or bash: Two different shells, they do what ms-dos can't.
  2. gawk or awk: awk is a tool that looks for a specified pattern and performs a specified action -- very powerful tool. gawk is gnu-awk, slightly different from awk in capabilities.
  3. comm: Find what is common between two files. Actually has options which can invert the output, for example find all things in file1 that do not exist in file2.
  4. grep: "group regular expression" something or the other :-), anyway what it does is look for strings in a file with great amount of flexibility using unix "regular expressions".
Steps: 
The following are a generic set of steps: 
  1. Export from your scheduler to a text format: If you can set options on which fields are to be exported, then do so and include the minimum fields. Go for tab delimited if possible. The end requirement for convdb is a record-by-record format, meaning each line contains details for one single event. If the export is already in this form you will have less trouble -- maybe no trouble. The export may also output in a field-by-field format in which case each line contains info about one field. In this case you have to literally "compile" a record from the values of the fields, which is a troublesome sometimes.
  2. Analyze the text file: See how it is formatted. There are typically two types of formats. One is record by record, meaning that all details for an event are in one single line of the file, the next line is the next event and so on.
  3. Write a batch file in win95 or a script in ksh/bash to process the export file: Depending on the fact whether the export file was in record-by-record or field-by-field format you will have to write slightly different scripts. Note: this step has to take the next step into consideration.

  4. For a  record-by-record format just slap a convdb header on top of the file. for example: 
    %m/%d%y <tab> %h:%i <tab> %v 
    This should be sufficient to convert a file with records that appear in the following manner: 
    1/29/98    12:30    Lunch 
     
      If the events are not in a record by record format, then you will have fields in separate lines. For example: 
      Start Time: 12:30 
      Start Date: 1/29/98 
      Event: Lunch 
    In this case you will have to use a tool such as awk/gawk to compile it into a record by record format. A sample awk file for this would look like: 
    ************************************************** 
    
    BEGIN { printf("%%m/%%d/%%y\t%%h:%%i\t%%v\n") } 
    
    # a BEGIN tag means: do this once at the beginning.
    # %% means print one % otherwise it is a keyword for printf 
    # a \t means tab , and \n means return character 
    
    /Start Time/ {time=$2} 
    
    # a / indicates the beginning and the end of a search string 
    # a $ followed by a number indicates the field 
    # so here it is the second field 
    
    /Start Date/ {date=$2} 
    /Event/ {printf ("%s\t%s\t%s\n",date, time, $2} 
    **************************************************
      
  5. Taking care of Synchronization: True synchronization can be done when deldba is finished. Right now you can at least avoid duplicating records in the following manner:
    1. Store the current convdb import file  as say import.bak
    2. the next time you convert from your scheduler,say the import file is called impor.current
    3. now there is unix tool called comm, this compares two files for commonality and is capable of extracting records unique to file1 only. So the command would be:

    4. comm -23 import.current import.bak 
      The output is entries in export.current that do not exist in export.bak. So, using this tool you have just circumvented the duplication problem. Once deldba is finished you would also look for records unique to export.bak, these are the ones to be deleted. 
      Pipe the comm command's output to a file. Add the header to this file and its set for import. For example: comm -23 import.current import.bak > import.uniq 
      then :       echo "m/%d%y <tab> %h:%i <tab> %v" > import.convdb 
      then :       type import.uniq >> import.convdb 
      Note the double greater-than symbol >> this is not a typo, it means add to file's existing contents. 
    5. Run convdb on the import.convdb file

    6.