vipsi - developer documentation
Author: kio@little-bat.de
Keywords: vipsi shell script interpreter programming

# vipsi - shell and script language

## developer documentation

### documentation at little-bat.de

#### the beauty within

hi!

i'm an optical person with bad eyes and nearly-non-present memory. sometimes it really helps me if i follow the following design guides, and therefore i ask you that you try to follow them too to exactly that extent which does not disturb you ;-)

- in case you use non-7-bit-ascii characters: please use - if possible - utf-8 encoding.
  in case you don't use a utf-8 enabled editor: please don't "correct" the funny characters...
- all sources are formatted with 4 characters per tab.
- please indent loops, procedure bodies et.al.
  matching brackets should be either in the same row or column.
  i know that some people like it "more compact".
- use, as appropriate, the predefined data types.
- function names should start with an uppercase letter, except if they are very low-level procedures which are only used locally.
- variable and constant names should start with a lowercase letter.
- macros should be all uppercase.
- start each function definition with a comment which tells, what this function does, what it expects as input, what it supplies as output, which side effects it may imply, what it does when an error happens. ok, i'm sometimes pretty lazy here too, especially when i'm working on it or modifying it. and afterwards i'm too tired...
- keep it easy. don't be kewl. you'll regret it in a year or two.

#### class Var (not up to date documentation)

the class Var is part of the project lib-kio!

this class implements all the variables used in vipsi scripts.

currently it supports numbers (double float), text (class String), proc (procedure bodies: class String) and lists. it has a full set of operators which work on nearly all data types. most notably they also work on lists; e.g. you can multiply all list items by a number value or a matching second list to do vector operations.

#### class String (not up to date documentation)

the class Var is part of the project lib-kio!

this class is used by class Var for all text variables, proc variables (procedure bodies) and variable names. it is also used in various libraries for passing text, since this is the natural format for text representation in vipsi, because text read from variables has this format and requires to have this format whenever stored back in variables.

### predefined data types

file standard_types.h defines a lot of basic data types to shorten the source. there is no real need to use them, but it would be nice. this is not a complete list, just the most commonly used:

uchar | unsigned char | unsigned character
ushort | unsigned short | unsigned short integer
ulong | unsigned long | unsigned long integer
uint | unsigned int | unsigned integer
str | char * | indicates a 0-terminated c-string
cstr | char const * | 0-terminated c-string constant
ptr | char * | pointer to just a char
cptr | char const * | pointer to a constant char
uptr | uchar * | pointer to just an unsigned char or length-prefixed pascal-style string
cuptr | uchar const * | pointer to unsigned constant characters
OSErr | short or int | (operating system) error code

you probably recognize how a prefixed u stands for unsigned and c stands for constant.

project specific and project-independent general-use classes are as follows:

String | class String | text or binary data
cString | const class String | constant text or binary data
Var | class String | text or binary data
cVar | const class String | constant text or binary data

the utf-8 character set handling module utf-8.h defines a lot of own data types:

UCS4Char | unsigned long | single character, UCS-4 encoded
UCS2Char | unsigned short | single character, UCS-2 encoded
UCS1Char | unsigned char | single character, UCS-1 (ISO Latin-1) encoded
UTF8Char | char const * | single character, UTF-8 encoded
UTF8CharPtr | char * | pointer to UTF-8 character
UTF8Str | char * | c-string, UTF-8 encoded
cUTF8CharPtr | char const * | pointer to UTF-8 character in const string this is not same as const UTF8CharPtr!

any class or data type missing? send me a mail and i'll add it.

### uchar* VScript::Value ( uchar* ip, Var*&, int prio )

#### how does `VScript::Value()` work?

##### instruction pointer

like all the associated functions it takes the current instruction pointer as it's first argument and returns the updated pointer. and second: like all the associated functions it refuses work if the global errno is set. this allows you frequently to delay error testing, because you can combine a whole bunch of calls before you really must test if something went wrong.

##### Var*&

the result of the expression which is currently executed is returned in a supplied variable. first you must allocate a variable as temporary data cell where the result can be stored in, and then you must pass a pointer to this temp cell by reference.

why so complicated? vipsi does not know about reference variables. but frequently you need a reference to a variable or it saves you a lot of time if you don't have to copy the result into the supplied temp cell. on the other hand, frequently you can't return a reference to a result but you can only return it by value. have a look at the following example:

    	fooVar = 45;

on the left side of the operator `=` stands a variable. when Value() is called to parse the expression it is required that it returns a reference to that variable. the current contents of fooVar is of no interest.

on the right side of the operator `=` stands a number literal (constant). when Value() is called to parse the right side of the expression it cannot return a reference to a variable with this value because there is simply no such variable. you could circumvent this in a lots of other ways, but i've done it this way:

if the result cannot be returnd by reference, the value is stored in the supplied temp cell (as accessible by the passed pointer) and the pointer is left to point to this temp cell.

if the result can be returned by reference, the supplied temp cell is ignored and the passed-by-reference pointer is set to point to the result.

whether the returned result is a value or a reference can be tested by the caller, if he compares the variable pointer with it's initial value, the address of the temp cell. so he can see whether he got a reference when he needed one or not and he can see, whether he must duplicate the passed result from the referenced variable or can move the data from the temp cell to the target. or he can already supply the target as temp cell. sorry, he can't. this will lead to some dangerous conflicts.

##### prio

Value() calls itself recursively.

an expression is made out of multiple subexpressions connected by operators:

        <term>  =  <term> <op> <term>

where you recursively can replace a term again by `<term> <op> <term>`.

when you evaluate an expression with multiple operators, you have to obey operator precedence. the precedence of the currently pending operator is passed in prio. imagine the following expression:

        55 + 66 * 77;

- Value() is called with an instruction pointer pointing to the number `55`.
- Value() reads the number literal `55` which is the simplest form of a term.
  now it expects an operator.
- Value() reads the token for `+` which it recognizes as an operator.
  now it has to decide whether the 'pending' operator is of higher or same priority.
  if it is, Value() must put pack the operator and return.
  but you have probably called Value() with a prio low enough: `pAny` and Value() will resume, making the operator `+` it's currently pending operator. the enumeration of operator priorities is in the file vscript.cp.
- Value() now needs to evaluate the right side of the operator. for this purpose it calls itself recursively with the precedence of the current operator `+`.
  - Value() is called with an instruction pointer pointing to the number `66` and a prio corresponding to operator `+`.
  - Value() reads the number literal `66` which is the simplest form of a term.
    now it expects an operator.
  - Value() reads the token for `*` which it recognizes as an operator.
    now it has to decide whether the 'pending' operator is of higher or same priority.
    if it is, Value() must put pack the operator and return.
    but the passed prio is lower, and so it resumes, making operator `*` it's currently pending operator.
  - Value() now needs to evaluate the right side of the operator. for this purpose it calls itself recursively with the precedence of the current operator `*`.
    - Value() is called with an instruction pointer pointing to the number `77` and a prio corresponding to operator `*`.
    - Value() reads the number literal `77` which is the simplest form of a term.
      now it expects an operator.
    - Value() reads the token for `;` which is not an operator.
      Value() decides, that this is either an error or that the expression ends here. because it does not know much about the context where it was called from, it simply puts back the unrecognized token and returns the already evaluated term `77`, leaving it to the caller to decide whether he can handle the unrecognized token `;`.
  - Value() gets a result, all seems to be fine and calculates the current term, which is `66*77`.
    huuh, i have evaluated something and it's value is `5082`!
    now it expects an operator.
  - Value() reads the token for `;` which is not an operator.
    Value() decides, that this is either an error or that the expression ends here. because it does not know much about the context where it was called from, it simply puts back the unrecognized token and returns the already evaluated term `5082`, leaving it to the caller to decide whether he can handle the unrecognized token `;`.
- Value() gets the required right-side result, all seems to be fine and calculates the current term, which is `55+5082`.
  huuh, i have evaluated something and it's value is `5132`!
  now it expects an operator.
- Value() reads the token for `;` which is not an operator.
  Value() decides, that this is either an error or that the expression ends here. because it does not know much about the context where it was called from, it simply puts back the unrecognized token and returns the already evaluated term `5132`, leaving it to the caller to decide whether he can handle the unrecognized token `;`.

the initial caller gets back the value `5132` and resumes parsing. he will run into the `;` and probably decide that it's ok.

### map of project files

updated 2003-08-15

    vipsi/                          vipsi's folder
        execute.h                   the main execution loop:  VScript::Execute(…)
        main.cp                     main()
        mod_vipsi.c                 Apache module
        mod_vipsi.cp                Apache module
        Stream.cp                   class Stream to implement vipsi's i/o commands
        Stream.h
        SystemIO.cp                 platform specific file handling et.al.
        SystemIO.h
        token.cp                    the source text tokenizer: Tokenize()
        token.h                     token (word) definitions
        value.h                     the expression evaluator: VScript::Value(…)
        verrors.h                   additional error codes (included by libraries/kio/errors.h)
        version.h                   version file
        VScript.cp                  this class implements the script interpreter
        VScript.h

        examples/                   some example scripts
            test.vs                 syntax test suite

        ****INFO****/
            done.txt                completed tasks
            README.txt              basic notes on vipsi, compilation and installation
            todo.txt                pending tasks

        unix/                       files for installation on unix-style systems
            configure.in            input data for autoconf
            Makefile.in             input data for configure
            config.h.in             input data for configure
            configure               script to make a Makefile for your system
            linuxerrors.h           just required
            install.sh              required script
            config.sub              required script
            aclocal.m4              required script
            config.guess            required script
            shtool                  required script

        libraries/                  libraries included from project lib-kio
            compress/               data compression
            cstrings/               c-string library with ad-hoc string pool
            ustring/                class String for text and binary data strings
            sort/                   sort function template
            utf-8/                  utf-8 handling and text conversion library
            var/                    class Var for variables for interpreters et.al.
            kio/                    my all-times-needed header files
                abort.cp            aborting
                abort.h
                errors.cp           set and get errors
                errors.h            error codes and texts
                kio.h               the include-it-all header
                log.cp              message logging
                log.h
                nice_defines.h      frequently used simple functions
                standard_types.h    standard data types
            unix/
                file_utilities.cp   unix file utilities
                file_utilities.h
                os_utilities.cp     unix system call utilities
                os_utilities.h