README.zxid ########### <> <> 1 Who needs this? ================= ZXID project has currently (Aug 2006) two outputs libzxid:: A C library for supporting SAML 2.0, including federated Single Sign-On zxid:: A C program that implements a SAML Service Provider (SP) as a CGI script You need this if you are Web Master:: You want to enable SAML based Single Sign-On (SSO) to your web site. In this case you would use the zxid SP CGI script directly, only configuring it slightly. Perl Developer:: You can use the Net::SAML module to integrate SSO to your application and web site. Given the direct perl support this is easier than fully understanding the C interface. PHP Developer:: We expect to support functionality roughly equivalent to perl Net::SAML. It does not help that swig.org project does not officially support php5. Perhaps there is something the php5 community could do to fix this? (*** php4 vs. php5 penetration?) Web Developer:: You want to integrate SAML based SSO to your web site tool or product so that your customers can enjoy SSO enabled web sites. In this case you would study zxid.c for examples and use libzxid.a to implement the functionality in your own program. Identity Management hacker:: you need some building blocks: you will study libzxid and add to it, contributing to the project. ZXID Project has vastly more ambitious goals. See the ZXID Project chapter later in this document. 2 Installing ============ If you want to try ZXID out immediately, we recommend compiling the library and examples and installing one of the examples as a CGI script in an existing web server. See later chapters for more details. tar xvzf zxid-0.4.tgz cd zxid-0.4 # N.B. There is no configure script. The Makefile works for all supported platforms as is. make make perlmod # optional cd Net; make install # optional: install Net::SAML perl module cp zxid / # configure your web server to recognize zxid a CGI, e.g. mini_httpd -p 8443 -c zxid -S -E zxid.pem # Edit your /etc/hosts to contain 127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org # Point your browser to https://sp1.zxidsp.org:8443/zxid?o=E https://sp1.zxidsp.org:8443/zxid.pl?o=E # Perl version # Find an IdP to test with and configure it... 2.1 Prerequisites ----------------- This software depends on following packages: 1. zlib from zlib.net. Generally whatever comes with your distro is sufficient. 2. openssl-0.9.8b or later. See www.openssl.org. Generally openssl libraries distributed with most Linux distros are sufficient.<> 3. libcurl from http://curl.haxx.se/. I used version 7.15.5, but probably whatever ships with your distribution is fine. libcurl is needed for SOAP bindings and for fetching metadata. It needs to be compiled to support HTTPS.<> 4. HTTPS capable web server. For most trivial testing CGI support is needed. We recommend mini_httpd available from http://www.acme.com/software/mini_httpd/ Following additional packages are needed by developers who wish to build from scratch, including the code generation (the standard distribution includes the output of the code generation, so most people do not need these). a. gperf from gnu.org (only for build process when generating code) b. swig from swig.org (only for build process and only if you want scripting interfaces) c. perl from cpan.org (only for build process and only if you want to generate code from .sg) d. plaindoc from http://mercnet.pt/plaindoc/pd.html (only for build process, for code generation from .sg, and for documentation) e. mini_httpd from http://www.acme.com/software/mini_httpd/ (only for canned tutorial) Although technically not needed to build zxid, you will need an IdP to test against. We do not, at the time, supply one so you will need to find a third party. 2.2 Canned Tutorial: Running ZXID as CGI under mini_httpd --------------------------------------------------------- While zxid will run easily under Apache httpd (see <>), for sake of simplicity we first illustrate running it with mini_httpd(8), a very simple SSL capable web server by Jef Poskanzer. 2.2.1 Getting and installing mini_httpd ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can download the source for mini_httpd from http://www.acme.com/software/mini_httpd/ You should already have installed OpenSSL, or quite probably OpenSSL shipped with your distribution. If it is not located at /usr/local/ssl, the you need to edit the mini_httpd ~Makefile~ to indicate where it is. At any rate you need to uncomment all lines that start by SSL_ in the ~Makefile~. Then say make Now copy the mini_httpd binary somewhere in your path. 2.2.2 Running mini_httpd ~~~~~~~~~~~~~~~~~~~~~~~~ After building zxid, in zxid directory, run mini_httpd -p 8443 -c zxid -S -E zxid.pem where -p 8443 specifies the port to listen to -c zxid specifies that URL paths ending in "zxid" are CGI scripts -S specifies that https is to be used -E zxid.pem specifies the SSL certificate to use > N.B. The zxid.pem certificate and private key combo is shipped with zxid > for demonstration purposes. Obviously everybody who downloads zxid > has that private key, so there is no real security what-so-ever. For > production use, you must generate, or aquire, your own private > key-certificate pair (and keep the private key secret). See Certificates > chapter for further info. 2.2.3 Accessing ZXID ~~~~~~~~~~~~~~~~~~~~ Edit your /etc/hosts file so that the definition of localhost also includes sp1.zxidcommon.org and sp1.zxidsp.org domain names, e.g: 127.0.0.1 localhost sp1.zxidcommon.org sp1.zxidsp.org Point your browser to https://sp1.zxidsp.org:8443/zxid or if you do not want the common domain cookie check https://sp1.zxidsp.org:8443/zxid?o=E 2.2.4 Setting up an IdP ~~~~~~~~~~~~~~~~~~~~~~~ Currently zxid does not ship with an IdP (though the necessary protocol encoders and decoders are latently available in libzxid, should anyone wish to make an attempt to hack an IdP together). For you to test zxid, you will need to aquire an IdP from somewhere - any vendor whose product is SAML 2.0 certified will do. One possible source is http://symlabs.com/Products/SFIAM.html who have a free download. If you do not want to install an IdP yourself (even for testing), find someone who already runs one and ask if they would be willing to load the metadata of your zxid SP. Once you get your IdP up and running, you need to make sure it accepts the zxid SP in its Circle of Trust (CoT). This is done by placing the metadata of the SP in right place in the IdP product configuration. If your IdP supports automatic CoT management, just turn it on and chances are you are done.<> If not, you can obtain the zxid SP metadata (which is slightly different for each install so you can't just copy it from existing install) from https://sp1.zxidsp.org:8443/zxid?o=B This URL is the +well known location method+ metadata URL. It is also the SP +Entity ID+ or Provider ID, should the IdP product ask for this in its configuration. If the IdP product needs you to supply the metadata manually as an xml file, just point your web browser to the above URL and save to file. zxid SP, by default, has automatic fetching of IdP metadata enabled so there is no manual configuration step needed, provided that the IdP supports the well known location method. All SAML 2.0 certified IdP implementations must support it (but you may still need to enable it in configuration). However, you will need the Entity ID (Provider ID) of the IdP. This is the URL that the IdP uses for well known location method of metadata sharing. You may need to dig the IdP documentation or GUI for a while to find it. If you already have the IdP metadata as an xml file, open it and look for EntityDescriptor/entityID. If you already have the file, you can also import it manually by running following command ./zxid -import file:///path/to/idp-meta.xml But the preferred method still is just let the automatic method do its job. 2.2.5 Your first SSO ~~~~~~~~~~~~~~~~~~~~ 1. Start at https://sp1.zxidsp.org:8443/zxid or https://sp1.zxidsp.org:8443/zxid?o=E If you had common domin cookie already in place, and you are already logged in the IdP, the SSO may happen automatically (go to step 3). The automatic experience will be typical when you use SSO regularily for more than one web site (i.e. SP). However, if you get a screen titled "ZXID SP SSO", you need to paste the IdP's Entity ID to the supplied field and click "Login". If zxid SP already obtained the metadata for the IdP, you may also see a button specific for your IdP (and in this case there is no need to know the Entity ID anymore or paste anything). 2. Next step depends on the IdP product you are using. Usually a login screen will appear asking for user name and password. Supply these and login. You will need an account at the IdP. 3. For more slick IdPs, that's all you need to do and you will land right back at the zxid SP page titled "ZXID SP Management". > Congratulations, you have made your first SSO! However, some IdPs will pester you with additional questions and you will have to jump through their hoops. A typical question is whether you want to accept a federation. You do. Sometimes the federation question does not appear automatically and you need to figure out a way to create a federation in their user interface and how to get them to send you back to SP. Sometimes the word used is "account linking" instead of federation.<> 3 Configuring and Running ========================= ZXID ships with working demo configuration so you can run it right away and once you are familiar with the concepts, you can return to this chapter. ZXID uses a configuration file in hardwired path<> /var/zxid/zxid.conf for figuring out its parameters. If this file is not present, built-in default configuration is used. The built-in configuration will allow you to test features of ZXID, but should not be used in production because it uses default certificates and private keys. Obviously the demo private key is of public knowledge since it is distributed with the ZXID package, and as such it provides no privacy protection what-so-ever. For production use you MUST generate your own certificate and private key. Usually configuration of a system involves following tasks 1. Configure web server (see your web server documentation) a. HTTPS operation and TLS certificate. In the minimum you need the main site, but you may want to configure the Common Domain Cookie virtual host as well. b. Arrange for ZXID to be invoked. This could mean configuring zxid.x or zxid.pl to be recognized as a CGI script, or it could mean setting up your ~modperl~ or ~modphp~ system to call ZXID at the appropriate place. 2. Configure ZXID, including signing certificate and CoT with peer metadata a. generate or aquire certificate b. Obtain peer metadata (from their well known location) or enable +Instant CoT+ feature. 3. Configure CoT peers with your metadata. They can download your metadata from your well known location (which is the URL that is your entity ID). For this to happen you need to have web server and ZXID up and running. 3.1 Configuration Parameters ---------------------------- 3.1.1 zxidroot ~~~~~~~~~~~~~~ The root directory of ZXID configuration files and directories. By default this is /var/zxid and has following directories and files in it /var/zxid/ | +-- zxid.conf Main configuration file +-- pem/ Our certificates +-- cot/ Metadata of CoT partners (metadata cache) +-- ses/ Sessions `-- log/ Log files, pid files, and the like 3.1.2 pem ~~~~~~~~~ Directory that holds various certificates. The certificates have hardwired names that are not configurable. ca.pem:: Certification Authority certificates. These are used for validating any certificates received from peers (other sites on the CoT). The CA certificates may also be shipped to the peers to facilitate them validating our signatures. This is especially relevant if the certificate is issued by multilayer CA hierarchy where the peer may not have the intermediate CA certificates. sign-nopw-cert.pem:: The signing certificate AND private key (concatenated in one file). The private key MUST NOT be encrypted (there will not be any opportunity to supply decryption password). enc-nopw-cert.pem:: The encryption certificate AND private key (concatenated in one file). The private key MUST NOT be encrypted (there will not be any opportunity to supply decryption password). The signing certificate can be used as the encryption certificate. If encryption certificate is not specified it will default to signing certificate. In addition to the above certificates and private keys, you will need to configure your web server to use TLS or SSL certificates for the main site and the Common Domain site. We suggest the following naming ssl-nopw-cert.pem:: SSL or TLS certificate for main site. In order to avoid browser warnings, the CN field of this certificate should match the domain name of the site. The SSL certificate can be same as signing or encryption certificate. cdc-nopw-cert.pem:: SSL or TLS certificate for Common Domain Cookie introduction site. In order to avoid browser warnings, the CN field of this certificate should match the domain name of the site. The SSL certificate can be same as signing or encryption certificate. 3.1.3 cot ~~~~~~~~~ Directory that holds metadata of the Circle of Trust (CoT) partners. If +Instant CoT+ is enabled, this directory needs to be writable at run time. 3.2 Special or embedded compile (reduced functionality) ------------------------------------------------------- libzxid contains thousands of functions and any given application is unlikely to use them all. Thus the easiest, safest, no loss of functionality, way to reduce the footprint is to simply enable compiler and linker flags that support dead function elimination.<> If you need to squeeze zxid into as minimal space as possible, some functionality tradeoffs are supported. I stress that you should only attempt these tradeoffs once you are familiar with zxid and know what you are doing. The canned install instructions and tutorial walk throughs stop working if you omit significant functionality. 3.2.1 Compilation without OpenSSL ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Comment out the -DUSE_OPENSSL flag from CFLAGS in Makefile and recompile. This will cripple zxid from security perspective because it will no longer be able to verify or generate digital signatures. Unless your environment does not need trust and security, or you understand thoroughly how to provide trust and security by other means, it is a very bad idea to compile without OpenSSL. N.B. Compiling, or not, zxid with OpenSSL does not affect whether your web server will use SSL or TLS. Unless you know what you are doing, you should be using SSL at web server layer. Given that SSL is used at web server layer, the savings you would gain from compiling zxid without OpenSSL may be neglible if you use dynamic linking. 3.2.2 Compilation without libcurl ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Comment out the -DUSE_CURL flag from CFLAGS in Makefile and recompile. Disabling libcurl does not have adverse security implications: you only loose some functionality and depdending on your situation you may well be able to live without it. 1. Without libcurl, zxid can not act as a SOAP client. This has a few consequences a. Artifact profile for SSO is not supported because it needs SOAP to resolve the artifact. In most cases a perfectly viable alternative is to use POST profile for SSO. b. SOAP profiles for Single Logout and NameID management (aka defederation) are not supported. You can use the redirect profiles and get mostly the same functionality. 2. Automatic CoT metadata fetching using well known location method is not supported without libcurl. You can fetch the metadata manually, e.g. using web browser, and place it in /var/zxid/cot directory. If you want to manually control your Circle of Trust relationships, you probably want to do this anyway so loss of automatic functionality is a nonissue.<> 3. Web Services Client (WSC) functionality is not supported without libcurl. Effectively this is just another case of SOAP needed. If you have your own SOAP implementation, you may, at lesser automation, achieve much of the same functionality by calling the encoder and decoder functions manually. 3.2.3 Compiling without zlib (not supported) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ zlib is used mainly in redirect profiles. Since zlib foot print is small, we have made no supported provision to compile without it. If you hack something together, let us know. 4 Net::SAML Perl Module ======================= * perl CGI example: zxid.pl * using with modperl After building the main zxid tree, you can cd Net perl Makefile.PL make make test # Tests are extremely sparse at the moment make install This assumes you use the pregenerated Net/SAML_wrap.c and Net/SAML.pm files that we distribute. If you wish to generate these files from origin, you need to have SWIG installed and then say in main zxid directory make perlmod 5 PHP Integration ================= TBD using SWIG 6 Python Integration ==================== TBD using SWIG 7 Java Native API Integration ============================= TBD using SWIG 8 Native C API ============== The generated aspects of the native C API are in c/*-data.h, for example c/saml2-data.h Studying this file is very instructive. 8.1 C Data Structures --------------------- From .sg a header (.h) is generated. This header contains structs that represent the data of the elements. Each element and attribute generates its own node. Even trivial nodes like strings have to be kept this way because the nodes form basis of remembering the ordering of data. This ordering is needed for exclusive XML canonicalization, and thus for signature verification.<> Any missing data is represented by NULL pointer. Any repeating data is kept as a linked list, in reverse order of being seen in the data stream.<> Simple elements and all attributes are represented by simple string node (even if they are booleans or integers). 8.1.1 Handling Namespaces ~~~~~~~~~~~~~~~~~~~~~~~~~ An annoying feature of XML documents is that they have variable namespace prefixes. The namespace prefix for the unqualified elements is taken to be the one specified in target() directive of the .sg input. Name of an element in C code is formed by prefixing the element by the namespace prefix and an underscore. Attributes will only have namespace prefix if such was expressly specified in .sg input. However, pointer and length fields representing attributes take an "a" middlefix, thus giving a separate C identifier namespace for the attributes. When decoding, the actual namespace prefixes are recorded. The wire order encoder knows to use these recorded prefixes so that accurate canonicalization for XMLDSIG can be produced. The schema order encoder always uses the prefixes defined using target() directives in .sg files. The runtime notion of namespaces is handled by ~ns_tab~ field of the decoding and encoding context. It is initialized to contain all namespaces known by virtue of .sg declarations. The runtime assigned prefixes are held in a linked list hanging from ~n~ (next) field of ~struct zx_ns_s~. (*** more work needed here) The code generation creates a file such as c/saml2-ns.c which contains initialization for the table. The main program should point the ns_tab field of context as follows: struct zx_ns_s my_ns_tab[] = { #include "c/saml2-ns.c" { 0,0,0,0,0 } }; main { struct zx_ctx* ctx; ... ctx->ns_tab = my_ns_tab; } 8.1.2 Handling any and anyAttribute ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since our aim is to be lax in what we accept, every element can handle unexpected additional attributes as well as unexpected elements. Thus whether the schema specifies any or anyAttribute or not, we handle everything as if they were there. However, when attributes and elements are received out side of their expected context, they are simply trated as strings whith string names. This is true even for those attributes and elements that would be recognizable in their proper context. The any extension points, as well as some bookkeeping data are hidden inside ~ZX_ELEM_EXT~ macro. If you tinker with this macro, be sure you know what you are doing. If you want to add your own specific fields to all structs, redefining ~ZX_ELEM_EXT~ may be appropriate, but if you want to add more fields only to some specific structures, you can define a macro of form TPF_EEE_EXT and put in it whatever fields you want. These fields will be initialized to zero when the structure is created, but are not touched in any other way by the generated code. In particular, if some of your fields are pointers, it will be your responsibility to free them. The standard free functions will not understand to free them. See the data structure walking functions, below for one way to accomplish this. 8.1.3 Root data structure ~~~~~~~~~~~~~~~~~~~~~~~~~ The root data structure struct zx_root_s; is a special structure that has a field for evey top level recognizable element. 8.1.4 Per element data structures ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** TBW 8.1.5 Memory Allocation ~~~~~~~~~~~~~~~~~~~~~~~ After decoding all string data points directly into the input buffer, i.e. strings are NOT copied. Be sure to not free the input buffer until you are done processing the data structure. If you need to take a copy of the strings, you will need to walk the data structure as a post processing step and do your copies. This can be done using void TPF_dup_strs_len_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x); The structures are allocated via ZX_ZALLOC() macro, which by default calls zx_zalloc() function, which in turn uses system malloc(3). However, you can redefine the macro to use whatever other allocation scheme you desire. The generated libraries never free(3) memory. In many programming patterns, this is actually desireable: for example a CGI program can count on dying - the process exit(2) will free all the memory. If you need to free(3) the data structure, you will need to walk it using void TPF_free_len_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x, int free_strings); void zx_free_any(struct zx_dec_ctx* c, struct zx_note_s* n, int free_strs); The zx_free_any() works by having a gigantic switch statement that calls the appropriate specific free function. You can deep clone the data structure with void TPF_deep_clone_NS_EEE(struct zx_dec_ctx* c, struct TPF_NS_EEE_s* x, int dup_strings); struct zx_note_s* zx_clone_any(struct zx_dec_ctx* c, struct zx_note_s* n, int dup_strs); The zx_clone_any() works by having a gigantic switch statement that calls the appropriate specific free function. 8.2 Decoder as Recursive Descent Parser --------------------------------------- The entry point to the decoder is struct zx_root_s* zx_dec_root(struct zx_dec_ctx* c, int n); The decoding context holds pointer to the raw data and must be initialized prior to calling the decoder. The second argument specifies how many recognized elements are decoded before returning. Usually you would specify 1 to consume one top level element from the stream. The returned datastructure, ~struct zx_root_s~, contains one pointer for each type of top level element that can be recognized. The ~tok~ field of the returned value identifies the last top level element recognized and can be used to dispatch to correct request handler: struct TPF_root_s* x = TPF_dec_root(c, 1); switch (x->gg.g.tok) { case TPF_NS_EEE_ELEM: return process_EEE_req(x->NN_EEE); } When processing responses, it is generally already known which type of response you are expecting, so you can simply check for NULLness of the respective pointer in the returned data structure. Internally zx_dec_root() works much the same way: it scans a beginning of an element from the stream, looks up the token number corresponding to the element name, and switches on that, calling element specific decoder functions (see next section) to do the detailed processing. 8.2.1 Element Decoders ~~~~~~~~~~~~~~~~~~~~~~ For each recognizable element there is a function of form struct TPF_NS_EEE_s* zx_dec_NS_ELEM(struct zx_dec_ctx* c, int tok); This function works much the same way as ??? 8.2.2 Decoder Extension Points ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The generated code is instrumented with following macros ZX_ATTR_DEC_EXT(ss):: Extension point called just after decoding known attribute ZX_XMLNS_DEC_EXT(ss):: Extension point called just after decoding xmlns attribute ZX_UNKNOWN_ATTR_DEC_EXT(ss):: Extension point called just after decoding unknown attr ZX_START_DEC_EXT(x):: Extension point called just after decoding element name and allocating struct, but before decoding any of the attributes. ZX_END_DEC_EXT(x):: Extension point called just after decoding the entire element. ZX_START_BODY_DEC_EXT(x):: Extension point called just after decoding element tag, including attributes, but before decoding the body of the element. ZX_PI_DEC_EXT(pi):: Extension point called just after decoding processing instruction ZX_COMMENT_DEC_EXT(comment):: Extension point called just after decoding comment ZX_CONTENT_DEC(ss):: Extension point called just after decoding string content ZX_UNKNOWN_ELEM_DEC_EXT(elem):: Extension point called just after decoding unknown element Following macros are available to the extension points TPF:: Type prefix (as specified by -p during code generation) EL_NAME:: Namespaceful element name (NS_EEE) EL_STRUCT:: Name of the struct that describes the element EL_NS:: Namespace prefix of the element (as seen in input schema) EL_TAG:: Name of the element without any namespace qualification. 8.3 Exclusive Canonical Encoder ------------------------------- The encoder receives a C data structure and generates a gigantic string containing an XML document corresponding to the data structure and the input schemata. The XML document conforms to the rules of exclusinve XML canonicalization and hence is useful as input to XMLDSIG. One encoder is generated for each root node specified at the code generation. Often these encoders share code for interior nodes. The encoders allow two pass rendering. You can first use the length computation method to calculate the amount of storage needed and then call one of the rendering functions to actually render. Or if you simply have large enough buffer, you can render directly. 8.3.1 Length computation ~~~~~~~~~~~~~~~~~~~~~~~~ Compute length of an element (and its subelements). The XML attributes and elements are processed in schema order, although this should not really matter as length in wire order should be the same. int TPF_enc_len_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x); 8.3.2 Encoding in schema order ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Render element into string. The XML attributes and elements are processed in schema order. This is what you generally want for rendering new data structure to a string. The wo pointers are not used. char* TPF_enc_so_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x, char* p); 8.3.3 Encoding in wire order ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Render element into string. The XML attributes and elements are processed in wire order by chasing wo pointers. This is what you want for validating signatures on other people's XML documents. char* TPF_enc_wo_NS_EEE(struct zx_enc_ctx* c, struct TPF_NS_EEE_s* x, char* p); 8.4 Data Accessor Functions --------------------------- *** TBW 8.5 Memory Allocation and Free ------------------------------ *** TBW 8.6 Walking the data structure ------------------------------ *** TBW 8.9 Thread Safety ----------------- All generated libraries are designed to be thread safe, provided that the underlying libc APIs, such as malloc(3) are thread safe. 8 ZXID Project ============== Immediate goal: build a SAML 2.0 SP and ID-WSF 1.1 WSC Goals of ZXID project include * SOAP 1.1 support * SAML 2.0 compliance - SP role (highest short term priority) - IdP role * Liberty ID-FF 1.2 support - SP - IdP - SAML 1.1 * Liberty ID-WSF 1.1 support - Discovery bootstrap - Discovery WSC - ID-DAP WSC - ID-DAP WSP * Liberty ID-WSF 2.0 support - Discovery bootstrap - Discovery WSC - ID-DAP WSC - ID-DAP WSP 8.1 Project Layout ------------------ Following directory layout is used by the project. Many of the specified directories are used by intermediate outputs that are not distributed in tarball releases, but may or may no be present in CVS checkouts. zxid-0.xx | +-- xsd XML schema descriptions of protocols (not distributed) +-- sg Schema Grammar (.sg) descriptions of protocols +-- c C code generated from the Schema Grammar descriptions +-- tex Temporary files for document generation using PlainDoc (not distributed) +-- html HTML documentation generated using PlainDoc +-- review Publicly released announcements and documents (not distributed) +-- t Test scripts and expected test outputs `-- tmp Temporary files, such as actual test outputs 8.2 Protocol Encoders and Decoders ---------------------------------- The protocol encoders and decoders are generated automatically from the schema grammar (.sg) descriptions. This ensures accurate protocol implementation. While the output is strictly schema driven and correct, the decoders have some provisions to accept some deviations from strict spec (e.g. out of order elements are tolerated). However, one should note that XMLDSIG does not tolerate very much deviation, thus even if decoder accepts a slightly illfomed message, it is likely to fail in signature verification. There are three outputs from generation 1. Data structures describing the data (xx.h) 2. Encoder that linearizes the data structure to wire protocol (xx-enc.c) 3. Decoder that converts wire protocol byte stream to a data structure (xx-dec.c) 9 Code Generation Tools ======================= Main work horse of code generation is xsd2sg.pl, which serves multiple purposes 1. Build hashes of all declarations in .sg input. Each hash element consists of array of elements and attributes, as well as groups and attribute groups. The type of array element sis determined from prefix, per .sg rules. 2. Expand groups and attribute groups 3. Evaluate each element wrt its type and generate a. C data structures b. Decoder grammar c. Token descriptions for perfect hash and lexical analyzer d. Encoder C code The code to build hashes is intervowen in the code that generates .xsd from .sg. The rest of the generation happens in a function called generate(). Typical command line (to generate SAML 2.0 protocol engine) ~/plaindoc/xsd2sg.pl -d -gen saml2 -p zx_ \ -r saml:Assertion -r se:Envelope \ -S \ sg/saml-schema-assertion-2.0.sg \ sg/saml-schema-protocol-2.0.sg \ sg/xmldsig-core.sg \ sg/xenc-schema.sg \ sg/soap11.sg \ >/dev/null </dev/null >> To generate SAML 2.0 Metadata engine you would issue ~/plaindoc/xsd2sg.pl -d -gen saml2md -p zx_ \ -r md:EntityDescriptor -r md:EntitiesDescriptor \ -S \ sg/saml-schema-assertion-2.0.sg \ sg/saml-schema-metadata-2.0.sg \ sg/xmldsig-core.sg \ sg/xenc-schema.sg \ >/dev/null </dev/null >> 9.1 Special Support for Specific Programming Languages ------------------------------------------------------ While C code generation is the main output, and this can always be converted to other languages using SWIG, sometimes a more natural language interface can be built by directly generating it. We plan to enhance the code generation to do something like this. At least direct hash-of-hashes-of-arrays-of-hashes type datastructure generation for benefit of some scripting languages is planned. 10 ZXID SP ========== *** warning: not checked lately, may be wrong! <> *** add description of CGI fields 10 Certificates =============== TBD 11 License ========== Copyright (c) 2006 Sampo Kellomäki (sampo@iki.fi), All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. 12 FAQ ====== *** real user FAQs are still lacking. Maybe this stuff is perfect? 12.8 Author's Pet Peeves ------------------------ 1. What is Schema Grammar (.sg) and why are you using it? * Schema Grammar is a compact formal description of XML documents. It is mostly bidirectionally convertible to XML Schema (XSD) and captures the useful essence of most XML schemas. * Schema Grammars are intuitive and compact, often allowing the essence to be understood at glance, and even most complex cases being only about 50% of the volume of the corresponding XSD. * We use Schema Grammar descriptions because they are more human readable than XSD and still equally amenable to automated code generation. * Schema Grammar descriptions are usually converted using xsd2sg.pl, which is part of the PlainDoc distribution. * See http://mercnet.pt/plaindoc * N.B. You do not need xsd2sg.pl or PlainDoc if you just want to compile and use ZXID. 2. What is PlainDoc (.pd)? * PlainDoc is a document preparation system that uses intuitive plain text files with minimal markup to generate PDF and HTML outputs. * We use PlainDoc because it makes it easy to maintain documentation. * See http://mercnet.pt/plaindoc * N.B. You do not need PlainDoc if you just want to compile and use ZXID. 3. How come zxid is so heavy to compile? * SAML 2.0 and related specs have a lot of functionality and detail, even if you really only need 1% of it. We do not wish to arbitrate which functionality is best or most needed, so we simply provide it all. * A lot of the code is generated, thus the input for C compiler is well in excess of half a million lines of code (of which only about 6k were written by a human). * Some of the generated files are gigantic, e.g. Net/SAML/zxid_wrap.c is over 380k lines. Compiler has to process all of this as a single compilation unit. * gcc and gnu ld were, perhaps, not designed to process this large inputs efficiently. Often the implementation strategy of keeping everything in memory will cause a smaller machines to swap. * My 1GHz CPU, 256 MB RAM machine definitely swaps and thus takes about 45 minutes to compile all this stuff. * I recommend at least 1GB RAM and 3GHz CPU for development machine. On such machine, you should be able to build in about 10 min. 4. Why do you not use ./configure and GNU autoconf? * ~autoconf~ is not for everyone. World does not stop without ~autoconf~. Or indeed need ~autoconf~. It is Yet Another Dependency I Do Not Need (YADIDNN). * I find the GNU ~autoconf~ stuff much more difficult to understand than my own ~Makefile~. Why should I debug ~autoconf~ when I could spend the time debugging my ~Makefile~ or the actual code? * I find resolving problems much easier at source code and ~Makefile~ level than trying to debug a million line script generated by some system I do not understand (perhaps some hardcore ~autoconf~ advocate could try to convince me and educate me, but I doubt). * My policy is to only support systems I have first hand experience with, or I have trustworthy friends to rely on. It does not help me to have a system that tries to guess +gazillion irrellevant variables+ to an unpredictable state. It's much easier to stick to standards like POSIX and make sure you have predictable results from predictable inputs. * If the deterministic and predictable results are wrong, they can at least be debugged and fixed with a finite amount of work. * Supporting all relevant systems manually is not that much of work. The inhabitants of the orrelevant systems can support themselves, probably learning a great deal on the side. 12.9 Annoyances and improvement ideas ------------------------------------- There is a lot of commonality that is not leveraged, especially in the way service end points are chose given the metadata. The descriptors are nearly identical so casting them to one should work. Many of the SAML2 responses are nearly identical. Rather than construct them fully formally, we could have just one "SAML any response" function. Perhaps this could be supported by some schema grammar level aliasing feature: if an element derives from base type without adding anything at all of its own, we might as well only generate code for the base type. <ZXID Home

README ZXID

>> <>