veil_mainpage.c

00001 /* ----------
00002  * veil_mainpage.c
00003  *
00004  *      Doxygen documentation root for Veil
00005  *
00006  *      Copyright (c) 2005 - 2011 Marc Munro
00007  *      Author:  Marc Munro
00008  *  License: BSD
00009  *
00010  */
00011 
00012 
00013 /*! \mainpage Veil
00014 \version 9.1.0 (Stable))
00015 \section license License
00016 BSD
00017 \section intro_sec Introduction
00018 
00019 Veil is a data security add-on for Postgres.  It provides an API
00020 allowing you to control access to data at the row, or even column,
00021 level.  Different users will be able to run the same query and see
00022 different results.  Other database vendors describe this as a Virtual
00023 Private Database.
00024 
00025 \section Why Why do I need this?
00026 If you have a database-backed application that stores sensitive data,
00027 you will be taking at least some steps to protect that data.  Veil
00028 provides a way of protecting your data with a security mechanism
00029 within the database itself.  No matter how you access the database,
00030 whether you are a legitimate user or not, you cannot by-pass Veil
00031 without superuser privileges. 
00032 
00033 \subsection Advantages Veil Advantages
00034 By placing security mechanisms within the database itself we get a
00035 number of advantages:
00036 - Ubiquity.  Security is always present, no matter what application or
00037 tool is used to connect to the database.  If your application is
00038 compromised, your data is still protected by Veil.  If an intruder gets
00039 past your outer defences and gains access to psql, your data is still
00040 protected.
00041 - Single Security Policy and Implementation.  If you have N applications
00042 to secure, you have to implement your security policy N times.  With
00043 Veil, all applications may be protected by a single implementation.
00044 - Strength in Depth.  For the truly security conscious, Veil provides
00045 yet another level of security.  If you want strength in depth, with
00046 layers and layers of security like an onion, Veil gives you that extra
00047 layer.
00048 - Performance.  Veil is designed to be both flexible and efficient.
00049 With a good implementation it is possible to build access controls with
00050 a very low overhead, typically much lower than building the equivalent
00051 security in each application.
00052 - Cooperation.  The Veil security model is designed to cooperate with your
00053 applications.  Although Veil is primarily concerned with data access
00054 controls, it can also be used to provide function-level privileges.  If
00055 your application has a sensitive function X, it can query the database,
00056 through Veil functions, to ask the question, "Does the current user have
00057 execute_X privilege?".  Also, that privilege can be managed in exactly
00058 the same way as any other privilege.
00059 - Flexibility.  Veil is a set of tools rather than a product.  How you
00060 use it is up to you.
00061 
00062 \subsection Limitations Veil Limitations
00063 Veil can restrict the data returned by queries but cannot prevent the
00064 query engine itself from seeing restricted data.  In particular,
00065 functions executed during evaluation of the where clause of a query may
00066 be able to see data that Veil is supposed to restrict access to.
00067 
00068 As an example let's assume that we have a secured view, users, that
00069 allows a user to see only their own record.  When Alice queries the
00070 view, she will see this:
00071 
00072 \verbatim
00073 select * from users;
00074 
00075   userid  |   username
00076 ----------+-----------
00077    12345  |   Alice
00078 \endverbatim
00079 
00080 Alice should not be able to see any details for Bob or even, strictly
00081 speaking, tell whether there is an entry for Bob.  This query though:
00082 
00083 \verbatim
00084 select * from users 
00085 where 0 = 9 / (case username when 'Bob' then 0 else 1 end);
00086 \endverbatim
00087 
00088 will raise an exception if the where clause encounters the username
00089 'Bob'.  So Alice can potentially craft queries that will enable her to
00090 discover whether specific names appear in the database.
00091 
00092 This is not something that Veil is intended to, or is able to, prevent.
00093 Changes to the underlying query engine to attempt to plug such holes
00094 have been proposed, but they all have their limitations and are likely
00095 to lead to degraded performance.
00096 
00097 A more serious problem occurs if a user is able to create user defined
00098 functions as these can easily provide covert channels for leaking data.
00099 Consider this query:
00100 
00101 \verbatim
00102 select * from users where leak_this(username);
00103 \endverbatim
00104 
00105 Although the query will only return what the secured view allows it to,
00106 the where clause, if the function is deemed inexpensive enough, will see
00107 every row in the table and so will be able to leak supposedly protected
00108 data.  This type of exploit can be protected against easily enough by
00109 preventing users from defining their own functions, however there are
00110 postgres builtins that can be potentially be exploited in the same way.
00111 
00112 \subsection GoodNews The Good News
00113 
00114 The news is not all bad.  Although Veil can be circumvented, as shown
00115 above, a database protected by Veil is a lot more secure than one which
00116 is not.  Veil can provide extra security over and above that provided by
00117 your application, and in combination with a well designed application
00118 can provide security that is more than adequate for most commercial
00119 purposes.
00120 
00121 \section the-rest Veil Documentation
00122 - \subpage overview-page
00123 - \subpage API-page
00124 - \subpage Building
00125 - \subpage Demo 
00126 - \subpage Management
00127 - \subpage Esoteria
00128 - \subpage install
00129 - \subpage History
00130 - \subpage Feedback
00131 - \subpage Performance
00132 - \subpage Credits
00133 
00134 Next: \ref overview-page
00135 
00136 */
00137 /*! \page overview-page Overview: a quick introduction to Veil
00138 
00139 \section Overview-section Introduction
00140 The section introduces a number of key concepts, and shows the basic
00141 components of a Veil-protected system:
00142 - \ref over-views
00143 - \ref over-connections
00144 - \ref over-privs
00145 - \ref over-contexts
00146 - \ref over-funcs2
00147 - \ref over-roles
00148 
00149 \subsection over-views Secured Views and Access Functions
00150 Access controls are implemented using secured views and instead-of triggers.
00151 Users connect to an account that has access only to the secured views.
00152 For a table defined thus:
00153 \verbatim
00154 create table persons (
00155     person_id       integer not null,
00156     person_name     varchar(80) not null
00157 );
00158 \endverbatim
00159 The secured view would be defined something like this:
00160 \verbatim
00161 create view persons(
00162        person_id, person_name) as
00163 select person_id, person_name
00164   from persons
00165  where i_have_personal_priv(10013, person_id);
00166 \endverbatim
00167 
00168 A query performed on the view will return rows only for those persons
00169 where the current user has privilege 10013
00170 (<code>SELECT_PERSONS</code>).  We call the function
00171 <code>i_have_personal_priv()</code>, an access function.  Such
00172 functions are user-defined, and are used to determine whether the
00173 connected user has a specific privilege in any of a number of security
00174 contexts (see \ref over-contexts).  The example above is
00175 taken from the Veil demo application (\ref demo-sec) and 
00176 checks for privilege in the global and personal contexts.
00177 
00178 \subsection over-connections The Connected User and Connection Functions
00179 To determine a user's privileges, we have to know who that user is.
00180 At the start of each database session the user must be identified, and
00181 their privileges must be determined.  This is done by calling a
00182 connection function, eg:
00183 \verbatim
00184 select connect_person('Wilma', 'AuthenticationTokenForWilma');
00185 \endverbatim
00186 The connection function performs authentication, and stores the user's
00187 access privileges in Veil state variables.  These variables are then
00188 interrogated by the access functions used in the secured views.
00189 
00190 Prior to connection, or in the event of the connection failing, the
00191 session will have no privileges and will probably be unable to see any
00192 data.  Like access functions, connection functions are user-defined and 
00193 may be written in any language supported by PostgreSQL.
00194 
00195 \subsection over-privs Privileges
00196 Veil-based systems define access rights in terms of privileges.  A
00197 privilege is a named thing with a numerical value (actually, the name
00198 is kind of optional).
00199 
00200 An example will probably help.  Here is a definition of a privileges
00201 table and a subset of its data:
00202 \verbatim
00203 create table privileges (
00204     privilege_id    integer not null,
00205     privilege_name  varchar(80) not null
00206 );
00207 
00208 copy privileges (privilege_id, privilege_name) from stdin;
00209 10001   select_privileges
00210 10002   insert_privileges
00211 10003   update_privileges
00212 10004   delete_privileges
00213 . . .
00214 10013   select_persons
00215 10014   insert_persons
00216 10015   update_persons
00217 10016   delete_persons
00218 10017   select_projects
00219 10018   insert_projects
00220 10019   update_projects
00221 10020   delete_projects
00222 . . .
00223 10100   can_connect
00224 \.
00225 
00226 \endverbatim
00227 Each privilege describes something that a user can do.  It is up to the
00228 access and connection functions to make use of these privileges; the
00229 name of the privilege is only a clue to its intended usage.  In the
00230 example we might expect that a user that has not been given the
00231 <code>can_connect</code> privilege would not be able to authenticate
00232 using a connection function but this is entirely dependent on the
00233 implementation.
00234 
00235 \subsection over-contexts Security Contexts 
00236 
00237 Users may be assigned privileges in a number of different ways.  They
00238 may be assigned directly, indirectly through various relationships, or
00239 may be inferred by some means.  To aid in the discussion and design of a
00240 Veil-based security model we introduce the concept of security
00241 contexts, and we say that a user has a given set of privileges in a
00242 given context.  There are three types of security context:
00243 
00244  - Global Context.  This refers to privileges that a user has been given
00245    globally.  If a user has <code>select_persons</code> privilege in the
00246    global context, they will be able to select every record in the
00247    persons table.  Privileges in global context are exactly like
00248    database-level privileges: there is no row-level element to them.
00249 
00250  - Personal Context.  This context provides privileges on data that you
00251    may be said to own.  If you have <code>select_persons</code>
00252    privilege in only the personal context, you will only be able to
00253    select your own persons record.  Assignment of privileges in the
00254    personal context is often defined implicitly or globally, for all
00255    users, rather than granted explicitly to each user.  It is likely
00256    that everyone should have the same level of access to their own data
00257    so it makes little sense to have to explicitly assign the privileges
00258    for each individual user.
00259 
00260  - Relational Contexts.  These are the key to most row-level access
00261    controls.  Privileges assigned in a relational context are assigned
00262    through relationships between the connected user and the data to be
00263    accessed.  Examples of relational contexts include: assignments to
00264    projects, in which a user will gain access to project data only if
00265    they have been assigned to the project; and the management hierarchy
00266    within a business, in which a manager may have specific access to
00267    data about a staff member.  Note that determining a user's access
00268    rights in a relational context may require extra queries to be
00269    performed for each function call.  Your design should aim to minimise
00270    this.  Some applications may require several distinct relational 
00271    contexts.
00272 
00273 \subsection over-funcs2 Access Functions and Security Contexts
00274 Each access function will operate on privileges for a specific set of
00275 contexts.  For some tables, access will only be through global context.
00276 For others, it may be through global and personal as well as a number of
00277 different relational contexts.  Here, from the demo application, are a
00278 number of view definitions, each using a different access function that
00279 checks different contexts.
00280 \verbatim
00281 create view privileges(
00282        privilege_id,
00283        privilege_name) as
00284 select privilege_id,
00285        privilege_name
00286 from   privileges
00287 where  i_have_global_priv(10001);
00288 
00289 . . .
00290 
00291 create view persons(
00292        person_id,
00293        person_name) as
00294 select person_id,
00295        person_name
00296 from   persons
00297 where  i_have_personal_priv(10013, person_id);
00298 
00299 . . .
00300 
00301 create view projects(
00302        project_id,
00303        project_name) as
00304 select project_id,
00305        project_name
00306 from   projects
00307 where  i_have_project_priv(10017, project_id);
00308 
00309 . . .
00310 
00311 create view assignments (
00312        project_id,
00313        person_id,
00314        role_id) as
00315 select project_id,
00316        person_id,
00317        role_id
00318 from   assignments
00319 where  i_have_proj_or_pers_priv(10025, project_id, person_id);
00320 \endverbatim
00321 
00322 In the <code>privileges</code> view, we only check for privilege in the
00323 global context.  This is a look-up view, and should be visible to all
00324 authenticated users.
00325 
00326 The <code>persons</code> view checks for privilege in both the global
00327 and personal contexts.  It takes an extra parameter identifying the
00328 person who owns the record.  If that person is the same as the connected
00329 user, then privileges in the personal context may be checked.  If not,
00330 only the global context applies.
00331 
00332 The <code>projects</code> view checks global and project contexts.  The
00333 project context is a relational context.  In the demo application, a
00334 user gains privileges in the project context through assignments.  An
00335 assignment is a relationship between a person and a project.  Each 
00336 assignment record has a role.  This role describes the set of privileges
00337 the assignee (person) has within the project context.
00338 
00339 The <code>assignments</code> view checks all three contexts (global,
00340 personal and project).  An assignment contains data about a person and a
00341 project so privileges may be acquired in either of the relational
00342 contexts, or globally.
00343 
00344 \subsection over-roles Grouping Privileges by Roles
00345 Privileges operate at a very low-level.  In a database of 100 tables,
00346 there are likely to be 500 to 1,000 privileges in use.  Managing
00347 users access at the privilege level is, at best, tedious.  Instead, we
00348 tend to group privileges into roles, and assign only roles to individual
00349 users.  Roles act as function-level collections of privileges.  For
00350 example, the role <code>project-readonly</code> might contain all of the
00351 <code>select_xxx</code> privileges required to read all project data.
00352 
00353 A further refinement allows roles to be collections of sub-roles.
00354 Defining suitable roles for a system is left as an exercise for the
00355 reader.
00356 
00357 Next: \ref API-page
00358 
00359 */
00360 /*! \page API-page The Veil API
00361 \section API-sec The Veil API
00362 This section describes the Veil API.  It consists of the following
00363 sections
00364 
00365 - \ref API-intro
00366 - \subpage API-variables
00367 - \subpage API-simple
00368 - \subpage API-bitmaps
00369 - \subpage API-bitmap-arrays
00370 - \subpage API-bitmap-hashes
00371 - \subpage API-int-arrays
00372 - \subpage API-serialisation
00373 - \subpage API-control
00374 
00375 Note that all veil objects are placed in the veil schema.
00376 
00377 \section API-intro Veil API Overview
00378 Veil is an API that simply provides a set of state variable types, and 
00379 operations on those variable types, which are optimised for privilege
00380 examination and manipulation.
00381 
00382 The fundamental data type is the bitmap.  Bitmaps are used to
00383 efficiently record and test sets of privileges.  Bitmaps may be combined
00384 into bitmap arrays, which are contiguous groups of bitmaps indexed by
00385 integer, and bitmap hashes which are non-contiguous and may be indexed
00386 by text strings.
00387 
00388 In addition to the bitmap-based types, there are a small number of
00389 support types that just help things along.  If you think you have a case
00390 for defining a new type, please 
00391 \ref Feedback "contact" 
00392 the author.
00393 
00394 Next: \ref API-variables
00395 */
00396 /*! \page API-variables Variables
00397 Veil variables exist to record session and system state.  They retain
00398 their values across transactions.  Variables may be defined as either
00399 session variables or shared variables.
00400 
00401 All variables are referenced by name; the name of the variable is
00402 passed as a text string to Veil functions.
00403 
00404 Session variables are private to the connected session.  They are
00405 created when first referenced and, once defined, their type is set for
00406 the life of the session.
00407 
00408 Shared variables are global across all sessions.  Once a shared variable
00409 is defined, all sessions will have access to it.  Shared variables are
00410 defined in two steps.  First, the variable is defined as shared, and
00411 then it is initialised and accessed in the same way as for session
00412 variables.  Note that shared variables should only be created within
00413 \ref API-control-registered-init or \ref API-control-init.
00414 
00415 Note that bitmap refs and bitmap hashes may not be stored in shared
00416 variables.
00417 
00418 The following types of variable are supported by Veil, and are described
00419 in subsequent sections:
00420 - integers 
00421 - ranges
00422 - bitmaps
00423 - bitmap refs
00424 - bitmap arrays
00425 - bitmap hashes
00426 - integer arrays
00427 
00428 The following functions comprise the Veil variables API:
00429 
00430 - <code>\ref API-variables-share</code>
00431 - <code>\ref API-variables-var</code>
00432 
00433 Note again that session variables are created on usage.  Their is no
00434 specific function for creating a variable in the variables API.  For an
00435 example of a function to create a variable see \ref API-bitmap-init.
00436 
00437 \section API-variables-share share(name text)
00438 \verbatim
00439 function veil.share(name text) returns bool
00440 \endverbatim
00441 
00442 Implemented by C function veil_share(), this is used to define a
00443 specific variable as being shared.  A shared variable is accessible to
00444 all sessions and exists to reduce the need for multiple copies of
00445 identical data.  For instance in the Veil demo, role_privileges are
00446 recorded in a shared variable as they will be identical for all
00447 sessions, and to create a copy for each session would be an unnecessary
00448 overhead.  This function should only be called from 
00449 \ref API-control-registered-init or \ref API-control-init. 
00450 
00451 \section API-variables-var veil_variables()
00452 \verbatim
00453 function veil.veil_variables() returns setof veil_variable_t
00454 \endverbatim
00455 
00456 This function, implemented by C function veil_variables(), returns a
00457 description for each variable known to the session.  It provides the
00458 name, the type, and whether the variable is shared.  It is primarily
00459 intended for interactive use when developing and debugging Veil-based
00460 systems.
00461 
00462 Next: \ref API-simple
00463 */
00464 /*! \page API-simple Basic Types: Integers and Ranges
00465 
00466 Veil's basic types are those that do not contain repeating groups
00467 (arrays, hashes, etc).  
00468 
00469 Ranges, implemented by the type \ref veil_range_t,
00470 consist of a pair of values and are generally used to initialise the
00471 bounds of array and bitmap types.
00472 
00473 \anchor veil_range_t \ref veil_range_t is defined as:
00474 
00475 \verbatim
00476 create type veil.veil_range_t as (
00477     min  int4,
00478     max  int4
00479 );
00480 \endverbatim
00481 
00482 Ranges may not contain nulls.
00483 
00484 The int4 type is used to record a simple nullable integer.  This is
00485 typically used to record the id of the connected user in a session.
00486 
00487 The following functions comprise the Veil basic types API:
00488 
00489 - <code>\ref API-basic-init-range</code>
00490 - <code>\ref API-basic-range</code>
00491 - <code>\ref API-basic-int4-set</code>
00492 - <code>\ref API-basic-int4-get</code>
00493 
00494 \section API-basic-init-range init_range(name text, min int4, max int4)
00495 \verbatim
00496 function veil.init_range(name text, min int4, max int4) returns int4
00497 \endverbatim
00498 
00499 This, implemented by veil_init_range() defines a range, and returns the
00500 extent of that range.
00501 
00502 \section API-basic-range range(name text)
00503 \verbatim
00504 function veil.range(name text) returns veil.range_t
00505 \endverbatim
00506 
00507 This, implemented by C function veil_range() returns the contents
00508 of a range.  It is intended primarily for interactive use.
00509 
00510 \section API-basic-int4-set int4_set(name text, value int4)
00511 \verbatim
00512 function veil.int4_set(name text, value int4) returns int4
00513 \endverbatim
00514 
00515 Sets an int4 variable to a value, returning that same value.  It is
00516 implemented by C function veil_int4_set().
00517 
00518 \section API-basic-int4-get int4_get(name text)
00519 \verbatim
00520 function veil.int4_get(name text) returns int4
00521 \endverbatim
00522 
00523 Returns the value of the int4 variable given by name.  Implemented by C
00524 function veil_int4_get().
00525 
00526 
00527 Next: \ref API-bitmaps
00528 */
00529 /*! \page API-bitmaps Bitmaps and Bitmap Refs
00530 Bitmaps are used to implement bounded sets.  Each bit in the bitmap may
00531 be on or off representing presence or absence of a value in the set.
00532 Typically bitmaps are used to record sets of privileges.
00533 
00534 A bitmap ref is a variable that may temporarily reference another
00535 bitmap.  These are useful for manipulating specific bitmaps within
00536 bitmap arrays or bitmap hashes.  All bitmap operations except for \ref
00537 API-bitmap-init may take the name of a bitmap ref instead of a bitmap.
00538 
00539 Bitmap refs may not be shared, and the reference is only accessible
00540 within the transaction that created it.  These restrictions exist to
00541 eliminate the possibility of references to deleted objects or to objects
00542 from other sessions.
00543 
00544 The following functions comprise the Veil bitmaps API:
00545 
00546 - <code>\ref API-bitmap-init</code>
00547 - <code>\ref API-bitmap-clear</code>
00548 - <code>\ref API-bitmap-setbit</code>
00549 - <code>\ref API-bitmap-clearbit</code>
00550 - <code>\ref API-bitmap-testbit</code>
00551 - <code>\ref API-bitmap-union</code>
00552 - <code>\ref API-bitmap-intersect</code>
00553 - <code>\ref API-bitmap-bits</code>
00554 - <code>\ref API-bitmap-range</code>
00555 
00556 \section API-bitmap-init init_bitmap(bitmap_name text, range_name text)
00557 \verbatim
00558 function veil.init_bitmap(bitmap_name text, range_name text) returns bool
00559 \endverbatim
00560 This is used to create or resize a bitmap.  The first parameter provides
00561 the name of the bitmap, the second is the name of a range variable that
00562 will govern the size of the bitmap.  It is implemented by C function
00563 veil_init_bitmap().
00564 
00565 \section API-bitmap-clear clear_bitmap(bitmap_name text)
00566 \verbatim
00567 function veil.clear_bitmap(bitmap_name text) returns bool
00568 \endverbatim
00569 This is used to clear (set to zero) all bits in the bitmap.  It is
00570 implemented by C function veil_clear_bitmap().
00571 
00572 \section API-bitmap-setbit bitmap_setbit(bitmap_name text, bit_number int4)
00573 \verbatim
00574 function veil.bitmap_setbit(bitmap_name text, bit_number int4) returns bool
00575 \endverbatim
00576 This is used to set a specified bit, given by bit_number in the bitmap
00577 identified by bitmap_name.  It is implemented by C function
00578 veil_bitmap_setbit().
00579 
00580 \section API-bitmap-clearbit bitmap_clearbit(bitmap_name text, bit_number int4)
00581 \verbatim
00582 function veil.bitmap_clearbit(bitmap_name text, bit_number int4) returns bool
00583 \endverbatim
00584 This is used to clear (set to zero) a specified bit in a bitmap.  It is
00585 implemented by C function veil_bitmap_clearbit().
00586 
00587 \section API-bitmap-testbit bitmap_testbit(bitmap_name text, bit_number int4)
00588 \verbatim
00589 function veil.bitmap_testbit(bitmap_name text, bit_number int4) returns bool
00590 \endverbatim
00591 This is used to test a specified bit in a bitmap.  It returns true if
00592 the bit is set, false otherwise.  It is implemented by C function
00593 veil_bitmap_testbit().
00594 
00595 \section API-bitmap-union bitmap_union(result_name text, bm2_name text)
00596 \verbatim
00597 function veil.bitmap_union(result_name text, bm2_name text) returns bool
00598 \endverbatim
00599 Form the union of two bitmaps with the result going into the first.
00600 Implemented by C function veil_bitmap_union().
00601 
00602 \section API-bitmap-intersect bitmap_intersect(result_name text, bm2_name text)
00603 \verbatim
00604 function veil.bitmap_intersect(result_name text, bm2_name text) returns bool
00605 \endverbatim
00606 Form the intersection of two bitmaps with the result going into the
00607 first.  Implemented by C function veil_bitmap_intersect().
00608 
00609 \section API-bitmap-bits bitmap_bits(bitmap_name text)
00610 \verbatim
00611 function veil.bitmap_bits(bitmap_name text) returns setof int4
00612 \endverbatim
00613 This is used to list all bits set within a bitmap.  It is primarily for
00614 interactive use during development and debugging of Veil-based systems.
00615 It is implemented by C function veil_bitmap_bits().
00616 
00617 \section API-bitmap-range bitmap_range(bitmap_name text)
00618 \verbatim
00619 function veil.bitmap_range(bitmap_name text) returns veil.range_t
00620 \endverbatim
00621 This returns the range, as a \ref veil_range_t, of a
00622 bitmap.  It is primarily intended for interactive use.  It is
00623 implemented by C function veil_bitmap_range().
00624 
00625 Next: \ref API-bitmap-arrays
00626 */
00627 /*! \page API-bitmap-arrays Bitmap Arrays
00628 A bitmap array is an array of identically-ranged bitmaps, indexed
00629 by an integer value.  They are initialised using two ranges, one for the
00630 range of each bitmap, and one providing the range of indices for the
00631 array.
00632 
00633 Typically bitmap arrays are used for collections of privileges, where
00634 each element of the collection is indexed by something like a role_id.
00635 
00636 The following functions comprise the Veil bitmap arrays API:
00637 
00638 - <code>\ref API-bmarray-init</code>
00639 - <code>\ref API-bmarray-clear</code>
00640 - <code>\ref API-bmarray-bmap</code>
00641 - <code>\ref API-bmarray-testbit</code>
00642 - <code>\ref API-bmarray-setbit</code>
00643 - <code>\ref API-bmarray-clearbit</code>
00644 - <code>\ref API-bmarray-union</code>
00645 - <code>\ref API-bmarray-intersect</code>
00646 - <code>\ref API-bmarray-bits</code>
00647 - <code>\ref API-bmarray-arange</code>
00648 - <code>\ref API-bmarray-brange</code>
00649 
00650 \section API-bmarray-init init_bitmap_array(bmarray text, array_range text, bitmap_range text)
00651 \verbatim
00652 function veil.init_bitmap_array(bmarray text, array_range text, bitmap_range text) returns bool
00653 \endverbatim
00654 Creates or resets (clears) the bitmap array named <code>bmarray</code>.
00655 The last two parameters are the names of ranges used to bound the
00656 dimensions of the array, and the range of bits within the array's
00657 bitmaps.  Implemented by C function veil_init_bitmap_array().
00658 
00659 \section API-bmarray-clear clear_bitmap_array(bmarray text)
00660 \verbatim
00661 function veil.clear_bitmap_array(bmarray text) returns bool
00662 \endverbatim
00663 Clear all bits in all bitmaps of the bitmap array named
00664 <code>bmarray</code>.  Implemented by C function veil_clear_bitmap_array().
00665 
00666 \section API-bmarray-bmap bitmap_from_array(bmref_name text, bmarray text, index int4)
00667 \verbatim
00668 function veil.bitmap_from_array(bmref_name text, bmarray text, index int4) returns text
00669 \endverbatim
00670 Place a reference into <code>bmref_name</code> to the bitmap identified
00671 by <code>index</code> in bitmap array <code>bmarray</code>.  Implemented
00672 by C function veil_bitmap_from_array().
00673 
00674 \section API-bmarray-testbit bitmap_array_testbit(bmarray text, arr_idx int4, bitno int4)
00675 \verbatim
00676 function veil.bitmap_array_testbit(bmarray text, arr_idx int4, bitno int4) returns bool
00677 \endverbatim
00678 Test a specific bit in a bitmap array.  Implemented by C function
00679 veil_bitmap_array_testbit().
00680 
00681 \section API-bmarray-setbit bitmap_array_setbit(bmarray text, arr_idx int4, bitno int4)
00682 \verbatim
00683 function veil.bitmap_array_setbit(bmarray text, arr_idx int4, bitno int4) returns bool
00684 \endverbatim
00685 Set a specific bit in a bitmap array.  Implemented by C function
00686 veil_bitmap_array_setbit().
00687 
00688 \section API-bmarray-clearbit bitmap_array_clearbit(bmarray text, arr_idx int4, bitno int4)
00689 \verbatim
00690 function veil.bitmap_array_clearbit(bmarray text, arr_idx int4, bitno int4) returns bool
00691 \endverbatim
00692 Clear a specific bit in a bitmap array.  Implemented by C function
00693 veil_bitmap_array_clearbit().
00694 
00695 \section API-bmarray-union union_from_bitmap_array(bitmap text, bmarray text, arr_idx int4)
00696 \verbatim
00697 function veil.union_from_bitmap_array(bitmap text, bmarray text, arr_idx int4) returns bool
00698 \endverbatim
00699 Union a bitmap with a specified bitmap from an array, with the result in
00700 the bitmap.    Implemented by C function
00701 veil_union_from_bitmap_array().  This is a faster shortcut for the
00702 following logical construction:
00703 
00704 \verbatim
00705 veil.bitmap_union(<bitmap>, veil.bitmap_from_array(<bitmap_array>, <index>))
00706 \endverbatim
00707 
00708 \section API-bmarray-intersect intersect_from_bitmap_array(bitmap text, bmarray text, arr_idx int4)
00709 \verbatim
00710 function veil.intersect_from_bitmap_array(bitmap text, bmarray text, arr_idx int4) returns bool
00711 \endverbatim
00712 Intersect a bitmap with a specified bitmap from an array, with the result in
00713 the bitmap.    Implemented by C function
00714 veil_intersect_from_bitmap_array().  This is a faster shortcut for the
00715 following logical construction:
00716 
00717 \verbatim
00718 veil.bitmap_intersect(<bitmap>, veil.bitmap_from_array(<bitmap_array>,<index>))
00719 \endverbatim
00720 
00721 \section API-bmarray-bits bitmap_array_bits(bmarray text, arr_idx int4)
00722 \verbatim
00723 function veil.bitmap_array_bits(bmarray text, arr_idx int4) returns setof int4
00724 \endverbatim
00725 Show all bits in the specific bitmap within an array.  This is primarily
00726 intended for interactive use when developing and debugging Veil-based
00727 systems.  Implemented by C function veil_bitmap_array_bits().
00728 
00729 \section API-bmarray-arange bitmap_array_arange(bmarray text)
00730 \verbatim
00731 function veil.bitmap_array_arange(bmarray text) returns veil_range_t
00732 \endverbatim
00733 Return the range of array indices, as a \ref veil_range_t, for the
00734 specified bitmap array.  Primarily for interactive use.  Implemented by
00735 C function veil_bitmap_array_arange().
00736 
00737 \section API-bmarray-brange bitmap_array_brange(bmarray text)
00738 \verbatim
00739 function veil.bitmap_array_brange(bmarray text) returns veil_range_t
00740 \endverbatim
00741 Show the range, as a \ref veil_range_t, of all bitmaps in the specified
00742 bitmap array.  Primarily for interactive use.  Implemented by
00743 C function veil_bitmap_array_range().
00744 
00745 
00746 Next: \ref API-bitmap-hashes
00747 */
00748 /*! \page API-bitmap-hashes Bitmap Hashes
00749 A bitmap hashes is a hash table of identically-ranged bitmaps, indexed
00750 by a text key.
00751 
00752 Typically bitmap hashes are used for sparse collections of privileges.
00753 
00754 Note that bitmap hashes may not be stored in shared variables as hashes
00755 in shared memory are insufficiently dynamic.
00756 
00757 The following functions comprise the Veil bitmap hashes API:
00758 
00759 - <code>\ref API-bmhash-init</code>
00760 - <code>\ref API-bmhash-clear</code>
00761 - <code>\ref API-bmhash-key-exists</code>
00762 - <code>\ref API-bmhash-from</code>
00763 - <code>\ref API-bmhash-testbit</code>
00764 - <code>\ref API-bmhash-setbit</code>
00765 - <code>\ref API-bmhash-clearbit</code>
00766 - <code>\ref API-bmhash-union-into</code>
00767 - <code>\ref API-bmhash-union-from</code>
00768 - <code>\ref API-bmhash-intersect-from</code>
00769 - <code>\ref API-bmhash-bits</code>
00770 - <code>\ref API-bmhash-range</code>
00771 - <code>\ref API-bmhash-entries</code>
00772 
00773 \section API-bmhash-init init_bitmap_hash(bmhash text, range text)
00774 \verbatim
00775 function veil.init_bitmap_hash(bmhash text, range text) returns bool
00776 \endverbatim
00777 Creates, or resets, a bitmap hash.  Implemented by
00778 C function veil_init_bitmap_hash().
00779 
00780 \section API-bmhash-clear clear_bitmap_hash(bmhash text)
00781 \verbatim
00782 function veil.clear_bitmap_hash(bmhash text) returns bool
00783 \endverbatim
00784 Clear all bits in all bitmaps of a bitmap hash.  Implemented by
00785 C function veil_clear_bitmap_hash().  Implemented by
00786 C function veil_clear_bitmap_hash().
00787 
00788 \section API-bmhash-key-exists bitmap_hash_key_exists(bmhash text, key text)
00789 \verbatim
00790 function veil.bitmap_hash_key_exists(bmhash text, key text) returns bool
00791 \endverbatim
00792 Determine whether a given key exists in the hash (contains a bitmap).
00793 Implemented by C function veil_bitmap_hash_key_exists().
00794 
00795 \section API-bmhash-from bitmap_from_hash(bmref text, bmhash text, key text)
00796 \verbatim
00797 function veil.bitmap_from_hash(bmref text, bmhash text, key text) returns text
00798 \endverbatim
00799 Generate a reference to a specific bitmap in a bitmap hash.  Implemented by
00800 C function veil_bitmap_from_hash().
00801 
00802 \section API-bmhash-testbit bitmap_hash_testbit(bmhash text, key text, bitno int4)
00803 \verbatim
00804 function veil.bitmap_hash_testbit(bmhash text, key text, bitno int4) returns bool
00805 \endverbatim
00806 Test a specific bit in a bitmap hash.  Implemented by
00807 C function veil_bitmap_hash_testbit().
00808 
00809 \section API-bmhash-setbit bitmap_hash_setbit(bmhash text, kay text, bitno int4)
00810 \verbatim
00811 function veil.bitmap_hash_setbit(bmhash text, key text, bitno int4) returns bool
00812 \endverbatim
00813 Set a specific bit in a bitmap hash.  Implemented by
00814 C function veil_bitmap_hash_setbit().
00815 
00816 \section API-bmhash-clearbit bitmap_hash_clearbit(bmhash text, key text, bitno int4)
00817 \verbatim
00818 function veil.bitmap_hash_clearbit(bmhash text, key text, bitno int4) returns bool
00819 \endverbatim
00820 Clear a specific bit in a bitmap hash.  Implemented by
00821 C function veil_bitmap_hash_clearbit().
00822 
00823 \section API-bmhash-union-into union_into_bitmap_hash(bmhash text, key text, bitmap text)
00824 \verbatim
00825 function veil.union_into_bitmap_hash(bmhash text, key text, bitmap text) returns bool
00826 \endverbatim
00827 Union a specified bitmap from a hash with a bitmap, with the result in
00828 the bitmap hash.  Implemented by C function
00829 veil_union_into_bitmap_hash().  This is a faster shortcut for the
00830 following logical construction:
00831 
00832 \verbatim
00833 veil.bitmap_union(veil.bitmap_from_hash(<bitmap_hash>, <key>), <bitmap>)
00834 \endverbatim
00835 
00836 \section API-bmhash-union-from union_from_bitmap_hash(bmhash text, key text, bitmap text)
00837 \verbatim
00838 function veil.union_from_bitmap_hash(bmhash text, key text, bitmap text) returns bool
00839 \endverbatim
00840 Union a bitmap with a specified bitmap from a hash, with the result in
00841 the bitmap.  Implemented by C function veil_union_from_bitmap_hash().
00842 This is a faster shortcut for the following logical construction:
00843 
00844 \verbatim
00845 veil.bitmap_union(<bitmap>, veil.bitmap_from_hash(<bitmap_array>, <key>))
00846 \endverbatim
00847 
00848 \section API-bmhash-intersect-from intersect_from_bitmap_hash(bitmap text, bmhash text, key text)
00849 \verbatim
00850 function veil.intersect_from_bitmap_hash(bitmap text, bmhash text, key text) returns bool
00851 \endverbatim
00852 Intersect a bitmap with a specified bitmap from a hash, with the result
00853 in the bitmap.  Implemented by C function
00854 veil_intersect_from_bitmap_hash().  This is a faster shortcut for the
00855 following logical construction:
00856 
00857 \verbatim
00858 veil.bitmap_intersect(<bitmap>, veil.bitmap_from_hash(<bitmap_array>, <key>))
00859 \endverbatim
00860 
00861 \section API-bmhash-bits bitmap_hash_bits(bmhash text, key text)
00862 \verbatim
00863 function veil.bitmap_hash_bits(bmhash text, key text) returns setof int4
00864 \endverbatim
00865 Show all bits in the specific bitmap within a hash.  This is primarily
00866 intended for interactive use when developing and debugging Veil-based
00867 systems.  Implemented by C function veil_bitmap_hash_bits().
00868 
00869 \section API-bmhash-range bitmap_hash_range(bmhash text)
00870 \verbatim
00871 function veil.bitmap_hash_range(bmhash text) returns veil_range_t
00872 \endverbatim
00873 Show the range, as a \ref veil_range_t, of all bitmaps in the hash.
00874 Primarily intended for interactive use.   Implemented by
00875 C function veil_bitmap_hash_range().
00876 
00877 \section API-bmhash-entries bitmap_hash_entries(bmhash text)
00878 \verbatim
00879 function veil.bitmap_hash_entries(bmhash text) returns setof text
00880 \endverbatim
00881 Show every key in the hash.  Primarily intended for interactive use.
00882 Implemented by C function veil_bitmap_hash_entries().
00883 
00884 Next: \ref API-int-arrays
00885 */
00886 /*! \page API-int-arrays Integer Arrays
00887 Integer arrays are used to store simple mappings of keys to values.  In
00888 the Veil demo (\ref demo-sec) they are used to record the extra privilege
00889 required to access person_details and project_details of each
00890 detail_type: the integer array being used to map the detail_type_id to
00891 the privilege_id.
00892 
00893 Note that integer array elements cannot be null.
00894 
00895 The following functions comprise the Veil int arrays API:
00896 
00897 - <code>\ref API-intarray-init</code>
00898 - <code>\ref API-intarray-clear</code>
00899 - <code>\ref API-intarray-set</code>
00900 - <code>\ref API-intarray-get</code>
00901 
00902 \section API-intarray-init init_int4array(arrayname text, range text)
00903 \verbatim
00904 function veil.init_int4array(arrayname text, range text) returns bool
00905 \endverbatim
00906 Creates, or resets the ranges of, an int array.  Implemented by
00907 C function veil_init_int4array().
00908 
00909 \section API-intarray-clear clear_int4array(arrayname text)
00910 \verbatim
00911 function veil.clear_int4array(arrayname text) returns bool
00912 \endverbatim
00913 Clears (zeroes) an int array.  Implemented by
00914 C function veil_clear_int4array().
00915 
00916 \section API-intarray-set int4array_set(arrayname text, idx int4, value int4)
00917 \verbatim
00918 function veil.int4array_set(arrayname text, idx int4, value int4) returns int4
00919 \endverbatim
00920 Set the value of an element in an int array.  Implemented by
00921 C function veil_int4array_set().
00922 
00923 \section API-intarray-get int4array_get(arrayname text, idx int4)
00924 \verbatim
00925 function int4array_get(arrayname text, idx int4) returns int4
00926 \endverbatim
00927 Get the value of an element from an int array.  Implemented by
00928 C function veil_int4array_get().
00929 
00930 Next: \ref API-serialisation
00931 */
00932 /*! \page API-serialisation Veil Serialisation Functions
00933 With modern web-based applications, database connections are often
00934 pooled, with each connection representing many different users.  In
00935 order to reduce the overhead of connection functions for such
00936 applications, Veil provides a serialisation API.  This allows session
00937 variables for a connected user to be saved for subsequent re-use.  This
00938 is particularly effective in combination with pgmemcache  
00939 http://pgfoundry.org/projects/pgmemcache/
00940 
00941 Only session variables may be serialised.
00942 
00943 The following functions comprise the Veil serialisatation API:
00944 
00945 - <code>\ref API-serialise</code>
00946 - <code>\ref API-deserialise</code>
00947 - <code>\ref API-serialize</code>
00948 - <code>\ref API-deserialize</code>
00949 
00950 \section API-serialise serialise(varname text)
00951 \verbatim
00952 function veil.serialise(varname text) returns text
00953 \endverbatim
00954 This creates a serialised textual representation of the named session
00955 variable.  The results of this function may be concatenated into a
00956 single string, which can be deserialised in a single call to
00957 veil_deserialise().  Implemented by C function veil_serialise().
00958 
00959 \section API-deserialise deserialise(stream text)
00960 \verbatim
00961 function veil.deserialise(stream text) returns text
00962 \endverbatim
00963 This takes a serialised representation of one or more variables as
00964 created by concatenating the results of veil_serialise(), and
00965 de-serialises them, creating new variables as needed and resetting their
00966 values to those they had when they were serialised.  Implemented by C
00967 function veil_deserialise().
00968 
00969 \section API-serialize serialize(varname text)
00970 \verbatim
00971 function veil.serialize(varname text) returns text
00972 \endverbatim
00973 Synonym for veil_serialise()
00974 
00975 \section API-deserialize deserialize(stream text)
00976 \verbatim
00977 function veil.deserialize(stream text) returns text
00978 \endverbatim
00979 Synonym for veil_deserialise()
00980 
00981 Next: \ref API-control
00982 */
00983 /*! \page API-control Veil Control Functions
00984 Veil generally requires no management.  The exception to this is when
00985 you wish to reset shared variables.  You may wish to do this because
00986 your underlying security definitions have changed, or because you have
00987 added new features.  In this case, you may use veil_perform_reset() to
00988 re-initialise your shared variables.  This function replaces the current
00989 set of shared variables with a new set in a transaction-safe manner.
00990 All current transactions will complete with the old set of variables in
00991 place.  All subsequent transactions will see the new set.
00992 
00993 The following functions comprise the Veil control functions API:
00994 
00995 - <code>\ref API-control-registered-init</code>
00996 - <code>\ref API-control-init</code>
00997 - <code>\ref API-control-reset</code>
00998 - <code>\ref API-version</code>
00999 
01000 \section API-control-registered-init registered initialisation functions
01001 
01002 A registered initialisation function is one which will be called from
01003 the standard veil \ref API-control-init function.  Such functions are
01004 responsible for defining and, usually, initialising shared variables.
01005 
01006 Initialisation functions may be written in any language supported by
01007 PostgreSQL, and must conform to the following function prototype:
01008 
01009 \verbatim
01010 function init_function(doing_reset bool) returns bool
01011 \endverbatim
01012 
01013 The <code>doing_reset</code> parameter will be set to true if we are
01014 completely resetting veil and redefining all of its variables.  In this
01015 case, we must declare and, probably, initialise shared variables prior to 
01016 any session initialisation actions.  The parameter will be false, if the
01017 function is solely being called to initialise a new session.  Check \ref
01018 demo-sec for an example. 
01019 
01020 Initialisation functions are registered by inserting their name into
01021 the configuration table <code>veil.veil_init_fns</code>.  The functions
01022 listed in this table are executed in order of the <code>priority</code>
01023 column.  Eg, to register <code>veil.init_roles()</code> to execute
01024 before <code>veil.init_role_privs()</code>, we would use the following
01025 insert statement:
01026 
01027 \verbatim
01028 insert into veil.veil_init_fns
01029        (fn_name, priority)
01030 values ('veil.init_roles', 1),
01031        ('veil.init_role_privs', 2);
01032 \endverbatim
01033 
01034 \section API-control-init veil_init(doing_reset bool)
01035 \verbatim
01036 function veil.veil_init(doing_reset bool) returns bool
01037 \endverbatim
01038 
01039 This function, implemented by the C function veil_init(), is reponsible
01040 for initialising each veil session.  The <code>doing_reset</code>
01041 parameter is true if we are to completely reset Veil, redefining all
01042 shared variables.
01043 
01044 The builtin implementation of veil_init() will call each registered
01045 initialisation function (see \ref API-control-registered-init) in turn.
01046 
01047 If no initialisation functions are registered, veil_init() raises an
01048 exception.
01049 
01050 As an alternative to registering initialisation functions, a Veil-based
01051 application may instead simply redefine veil.veil_init(), though this
01052 usage is deprecated.
01053 
01054 \section API-control-reset veil_perform_reset()
01055 \verbatim
01056 function veil.veil_perform_reset() returns bool
01057 \endverbatim
01058 This is used to reset Veil's shared variables.  It causes \ref
01059 API-control-init to be called.  Implemented by C function veil_perform_reset().
01060 
01061 \section API-version version()
01062 \verbatim
01063 function veil.version() returns text
01064 \endverbatim
01065 This function returns a string describing the installed version of
01066 veil.  Implemented by C function veil_version().
01067 
01068 Next: \ref Building
01069 
01070 */
01071 /*! \page Building Building a Veil-based secure database
01072 \section Build-sec Building a Veil-based secure database
01073 
01074 This section describes the steps necessary to secure a database using
01075 Veil.  The steps are:
01076 - \ref Policy
01077 - \ref Schemas
01078 - \ref Design
01079 - \ref Implementation
01080 - \ref Implementation2
01081 - \ref Implementation3
01082 - \ref Implementation4
01083 - \ref Testing
01084 
01085 \subsection Policy Determine your Policies
01086 
01087 You must identify which security contexts exist for your application,
01088 and how privileges should be assigned to users in those contexts.  You
01089 must also figure out how privileges, roles, and the assignment of roles
01090 to users are to be managed.  You must identify each object that is to be
01091 protected by Veil, identify the security contexts applicable for that
01092 object, and determine the privileges that will apply to each object in
01093 each possible mode of use.  Use the Veil demo application (\ref
01094 demo-sec) as a guide.
01095 
01096 For data access controls, typically you will want specific privileges
01097 for select, insert, update and delete on each table.  You may also want
01098 separate admin privileges that allow you to grant those rights.
01099 
01100 At the functional level, you will probably have an execute privilege for
01101 each callable function, and you will probably want similar privileges
01102 for individual applications and components of applications.  Eg, to
01103 allow the user to execute the role_manager component of admintool, you
01104 would probably create a privilege called
01105 <code>exec_admintool_roleman</code>.
01106 
01107 The hardest part of this is figuring out how you will securely manage
01108 these privileges.  A useful, minimal policy is to not allow anyone to
01109 assign a role that they themselves have not been assigned.
01110 
01111 \subsection Schemas Design Your Database-Level Security
01112 
01113 Veil operates within the security provided by PostgreSQL.  If you wish
01114 to use Veil to protect underlying tables, then those tables must not be
01115 directly accessible to the user.  Also, the Veil functions themselves,
01116 as they provide privileged operations, must not be accessible to user
01117 accounts. 
01118 
01119 A sensible basic division of schema responsibilities would be as follows:
01120 
01121 - An "owner" user will own the underlying objects (tables, views,
01122   functions, etc) that are to be secured.  Access to these objects will
01123   be granted only to "Veil".  The "owner" user will connect only when
01124   the underlying objects are to be modified.  No-one but a DBA will ever
01125   connect to this account, and generally, the password for this account
01126   should be disabled.
01127 
01128 - A "Veil" user will own all secured views and access functions (see
01129   \ref over-views).  Access to these objects will be granted to the
01130   "Accessor" user.  Like the "owner" user, this user should not be
01131   directly used except by DBAs performing maintenance.  It will also own
01132   the Veil API, ie this is the account where Veil itself will be
01133   installed.  Direct access to Veil API functions should not be granted
01134   to other users.  If access to a specific function is needed, it should
01135   be wrapped in a local function to which access may then be granted.
01136  
01137 - "Accessor" users are the primary point of contact.  These must have no
01138   direct access to the underlying objects owned by owner.  They will have
01139   access only to the secured views and access functions.  All
01140   applications may connect to these user accounts.
01141 
01142 \subsection Design Design Your Access Functions
01143 
01144 Provide a high-level view of the workings of each access function.  You
01145 will need this in order to figure out what session and shared variables
01146 you will need.  The following is part of the design from the Veil demo
01147 application:
01148 \verbatim
01149 Access Functions are required for:
01150 - Global context only (lookup-data, eg privileges, roles, etc)
01151 - Personal and Global Context (personal data, persons, assignments, etc)
01152 - Project and Global (projects, project_details)
01153 - All 3 (assignments)
01154 
01155 Determining privilege in Global Context:
01156 
01157 User has priv X, if X is in role_privileges for any role R, that has
01158 been assigned to the user.
01159 
01160 Role privileges are essentially static so may be loaded into memory as a
01161 shared variable.  When the user connects, the privileges associated with
01162 their roles may be loaded into a session variable.
01163 
01164 Shared initialisation code:
01165   role_privs ::= shared array of privilege bitmaps indexed by role.
01166   Populate role_privs with:
01167     select bitmap_array_setbit(role_privs, role_id, privilege_id)
01168     from   role_privileges;
01169 
01170 Connection initialisation code:
01171   global_privs ::= session privileges bitmap
01172   Clear global_privs and then initialise with:
01173     select bitmap_union(global_privs, role_privs[role_id])
01174     from   person_roles
01175     where  person_id = connected_user;
01176 
01177 i_have_global_priv(x):
01178   return bitmap_testbit(global_privs, x);
01179 
01180 \endverbatim 
01181 
01182 This gives us the basic structure of each function, and identifies what
01183 must be provided by session and system initialisation to support those
01184 functions.  It also allows us to identify the overhead that Veil imposes.
01185 
01186 In the case above, there is a connect-time overhead of one extra query
01187 to load the global_privs bitmap.  This is probably a quite acceptable
01188 overhead as typically each user will have relatively few roles.
01189 
01190 If the overhead of any of this seems too significant there are
01191 essentially 4 options:
01192 - Simplify the design.
01193 - Defer the overhead until it is absolutely necessary.  This can be done
01194   with connection functions where we may be able to defer the overhead
01195   of loading relational context data until the time that we first need
01196   it.
01197 - Implement a caching solution (check out pgmemcache).  Using an
01198   in-memory cache will save data set-up queries from having to be
01199   repeated.  This is pretty complex though and may require you to write
01200   code in C.
01201 - Suffer the performance hit.
01202 
01203 \subsection Implementation Implement the Initialisation Function
01204 
01205 Proper initialisation of veil is critical.  There are two ways to manage
01206 this.  The traditional way is to write your own version of \ref
01207 API-control-init, replacing the supplied version.  The newer, better,
01208 alternative is to register your own initialisation functions in the
01209 table veil.veil_init_fns, and have the standard \ref API-control-init,
01210 call them.  If there are multiple initialisation functions, they are
01211 called in order of their priority values as specified in the table
01212 <code>veil.veil_init_fns</code>.
01213 
01214 The newer approach has a number of advantages:
01215 
01216 - it fully supports the PostgreSQL extension mechanism, allowing
01217   extensions to be created and dropped;
01218 - it allows different security subsystems to have their own separate
01219   initialisation routines, allowing more modular code and better
01220   separation of responsibilities;
01221 - it is way cooler.
01222 
01223 Initialisation functions \ref API-control-init are critical elements.
01224 They will be called by automatically by Veil, when the first in-built
01225 Veil function is invoked.  Initialisation functions are responsible for
01226 three distinct tasks:
01227 
01228 - Initialisation of session variables
01229 - Initialisation of shared variables
01230 - Re-initialisation of variables during reset
01231 
01232 The boolean parameter to veil_init (which is passed to registered
01233 initialisation functions) will be false on initial session
01234 startup, and true when performing a reset (\ref API-control-reset).
01235 
01236 Shared variables are created using \ref API-variables-share.  This
01237 returns a boolean result describing whether the variable already
01238 existed.  If so, and we are not performing a reset, the current session
01239 need not initialise it.
01240 
01241 Session variables are simply created by using them.  It is worth
01242 creating and initialising all session variables to "fix" their data
01243 types.  This will prevent other functions from misusing them.
01244 
01245 If the boolean parameter to an initialisation fuction is true, then we
01246 are performing a memory reset, and all shared variables should be
01247 re-initialised.  A memory reset will be performed whenever underlying,
01248 essentially static, data has been modified.  For example, when new
01249 privileges have been added, we must rebuild all privilege bitmaps to
01250 accommodate the new values.
01251 
01252 \subsection Implementation2 Implement the Connection Functions
01253 
01254 The connection functions have to authenticate the connecting user, and
01255 then initialise the user's session.  
01256 
01257 Authentication should use a secure process in which no plaintext
01258 passwords are ever sent across the wire.  Veil does not provide
01259 authentication services.  For your security needs you should probably
01260 check out pgcrypto.
01261 
01262 Initialising a user session is generally a matter of initialising
01263 bitmaps that describe the user's base privileges, and may also involve
01264 setting up bitmap hashes of their relational privileges.  Take a look at
01265 the demo (\ref demo-sec) for a working example of this.
01266 
01267 \subsection Implementation3 Implement the Access Functions
01268 
01269 Access functions provide the low-level access controls to individual
01270 records.  As such, their performance is critical.  It is generally better
01271 to make the connection functions to more work, and the access functions
01272 less.  Bear in mind that if you perform a query that returns 10,000 rows
01273 from a table, your access function for that view is going to be called
01274 10,000 times.  It must be as fast as possible.
01275 
01276 When dealing with relational contexts, it is not always possible to keep
01277 all privileges for every conceivable relationship in memory.  When this
01278 happens, your access function will have to perform a query itself to
01279 load the specific data into memory.  If your application requires this,
01280 you should: 
01281 
01282 - Ensure that each such query is as simple and efficient as possible 
01283 - Cache your results in some way
01284 
01285 You may be able to trade-off between the overhead of connection
01286 functions and that of access functions.  For instance if you have a
01287 relational security context based upon a tree of relationships, you may
01288 be able to load all but the lowest level branches of the tree at connect
01289 time.  The access function then has only to load the lowest level branch
01290 of data at access time, rather than having to perform a full tree-walk.
01291 
01292 Caching can be very effective, particularly for nested loop joins.  If
01293 you are joining A with B, and they both have the same access rules, once
01294 the necessary privilege to access a record in A has been determined and
01295 cached, we will be able to use the cached privileges when checking for
01296 matching records in B (ie we can avoid repeating the fetch).
01297 
01298 \subsection Implementation4 Implement the views and instead-of triggers
01299 
01300 This is the final stage of implementation.  For every base table you
01301 must create a secured view and a set of instead-of triggers for insert,
01302 update and delete.  Refer to the demo (\ref demo-sec) for details of
01303 this.
01304 
01305 \subsection Testing Testing
01306 
01307 Be sure to test it all.  Specifically, test to ensure that failed
01308 connections do not provide any privileges, and to ensure that all
01309 privileges assigned to highly privileged users are cleared when a more
01310 lowly privileged user takes over a connection.  Also ensure that
01311 the underlying tables and raw veil functions are not accessible from
01312 user accounts.
01313 
01314 \section Automation Automatic code generation 
01315 
01316 Note that the bulk of the code in a Veil application is in the
01317 definition of secured views and instead-of triggers, and that this code
01318 is all very similar.  Consider using a code-generation tool to implement
01319 this. 
01320 
01321 Next: \ref Demo
01322 
01323 */
01324 /*! \page Demo A Full Example Application: The Veil Demo
01325 \section demo-sec The Veil Demo Application
01326 
01327 The Veil demo application serves two purposes:
01328 - it provides a demonstration of Veil-based access controls;
01329 - it provides a working example of how to build a secured system using Veil.
01330 
01331 This section covers the following topics:
01332 
01333 - \ref demo-install
01334 - \subpage demo-model
01335 - \subpage demo-security
01336 - \subpage demo-explore
01337 - \subpage demo-code
01338 - \subpage demo-uninstall
01339 
01340 \subsection demo-install Installing the Veil demo
01341 
01342 The veil_demo application, is packaged as an extension just like Veil
01343 itself, and is installed as part of the installation of Veil.  To create
01344 a database containing the veil_demo application:
01345 - create a new database
01346 - connect to that database
01347 - execute <code>create extension veil;</code>
01348 - execute <code>create extension veil_demo;</code>
01349 
01350 Next: \ref demo-model
01351 
01352 */
01353 /*! \page demo-model The Demo Database ERD, Tables and Views
01354 \section demo-erd The Demo Database ERD
01355 
01356 \image html veil_demo.png "The Veil Demo Database" width=10cm
01357 
01358 \section demo-tables Table Descriptions
01359 
01360 \subsection demo-privs Privileges
01361 
01362 This table describes each privilege.  A privilege is a right to do
01363 something.  Most privileges are concerned with providing access to
01364 data.  For each table, "X" there are 4 data privileges, SELECT_X, UPDATE_X,
01365 INSERT_X and DELETE_X.  There are separate privileges to provide access
01366 to project and person details, and there is a single function privilege,
01367 <code>can_connect</code>.
01368 
01369 \subsection demo-roles Roles
01370 
01371 A role is a named collection of privileges.  Privileges are assigned to
01372 roles through role_privileges.  Roles exist to reduce the number of
01373 individual privileges that have to be assigned to users, etc.  Instead
01374 of assigning twenty or more privileges, we assign a single role that
01375 contains those privileges.
01376 
01377 In this application there is a special role, <code>Personal
01378 Context</code> that contains the set of privileges that apply to all
01379 users in their personal context.  Since all users have the same personal
01380 context privileges, the demo application provides this role to all users
01381 implicitly; there is no need for it to be explicitly assigned.
01382 
01383 Assignments of roles in the global context are made through
01384 person_roles, and in the project (relational) context through
01385 assignments.
01386 
01387 \subsection demo-role-privs Role_Privileges
01388 
01389 Role privileges describe the set of privileges for each role.
01390 
01391 \subsection demo-role-roles Role_Roles
01392 
01393 This is currently unused in the Veil demo application.  Role roles
01394 provides the means to assign roles to other roles.  This allows new
01395 roles to be created as combinations of existing roles.  The use of this
01396 table is currently left as an exercise for the reader.
01397 
01398 \subsection demo-persons Persons
01399 
01400 This describes each person.  A person is someone who owns data and who
01401 may connect to the database.  This table should contain authentication
01402 information etc.  In the demo it just maps a name to a person_id.
01403 
01404 \subsection demo-projects Projects
01405 
01406 A project represents a real-world project, to which many persons may be
01407 assigned to work.
01408 
01409 \subsection demo-person-roles Person_Roles
01410 
01411 This table describes the which roles have been assigned to users in the
01412 global context.
01413 
01414 \subsection demo-assignments Assignments
01415 
01416 This describes the roles that have been assigned to a person on a
01417 specific project.  Assignments provide privilege to a user in the
01418 project context.
01419 
01420 \subsection demo-detail_types Detail_Types
01421 
01422 This is a lookup-table that describes general-purpose attributes that
01423 may be assigned to persons or project.  An example of an attribute for a
01424 person might be birth-date.  For a project it might be inception date.
01425 This allows new attributes to be recorded for persons, projects, etc
01426 without having to add columns to the table.
01427 
01428 Each detail_type has a required_privilege field.  This identifies the
01429 privilege that a user must have in order to be able to see attributes of
01430 the specific type.
01431 
01432 \subsection demo-person_details Person_Details
01433 
01434 These are instances of specific attributes for specific persons.
01435 
01436 \subsection demo-project-details Project_Details
01437 
01438 These are instances of specific attributes for specific projects.
01439 
01440 \section Demo-Views The Demo Application's Helper Views
01441 
01442 Getting security right is difficult.  The Veil demo provides a number of
01443 views that help you view the privileges you have in each context.
01444 
01445 - my_global_privs shows you the privileges you have in the global
01446   context
01447 - my_personal_privs shows you the privileges you have in the
01448   personal context
01449 - my_project_privs shows you the privileges you have for each project
01450   in the project context
01451 - my_privs shows you all your privileges in all contexts
01452 - my_projects shows you all the projects to which you have been assigned
01453 
01454 Using these views, access control mysteries may be more easily tracked
01455 down.
01456 
01457 Next: \ref demo-security
01458 
01459 */
01460 /*! \page demo-security The Demo Database Security Model
01461 \section demo-secmodel The Demo Database Security Model
01462 
01463 The Veil demo has three security contexts.
01464 
01465 - Personal Context applies to personal data that is owned by the
01466   connected user.  All users have the same privileges in personal
01467   context, as defined by the role <code>Personal Context</code>.
01468 - Global Context applies equally to every record in a table.  If a user
01469   has <code>SELECT_X</code> privilege in the global context, they will
01470   be able to select every record in <code>X</code>, regardless of
01471   ownership.  Privileges in global context are assigned through
01472   <code>person_roles</code>.
01473 - Project Context is a relational context and applies to project data.
01474   If you are assigned a role on a project, you will be given specific
01475   access to certain project tables.  The roles you have been assigned
01476   will define your access rights.
01477 
01478 The following sections identify which tables may be accessed in which
01479 contexts.
01480 
01481 \subsection demo-global-context The Global Context
01482 The global context applies to all tables.  All privilege checking
01483 functions will always look for privileges in the global context.
01484 
01485 \subsection demo-personal-context Personal Context
01486 The following tables may be accessed using rights assigned in the
01487 personal context:
01488 - persons
01489 - assignments
01490 - person_details
01491 
01492 \subsubsection demo-project-context Project Context
01493 The following tables may be accessed using rights assigned in the
01494 project context:
01495 - projects
01496 - assignments
01497 - project_details
01498 
01499 Next: \ref demo-explore
01500 
01501 */
01502 /*! \page demo-explore Exploring the Demo
01503 \section demo-use Exploring the Demo
01504 \subsection demo-connect Accessing the Demo Database
01505 Using your favourite tool connect to your veil_demo database.
01506 
01507 You will be able to see all of the demo views, both the secured views and
01508 the helpers.  But you will not initially be able to see any records:
01509 each view will appear to contain no data.  To gain some privileges you
01510 must identify yourself using the \ref demo-code-connect-person function.
01511 
01512 There are 6 persons in the demo.  You may connect as any of them and see
01513 different subsets of data.  The persons are
01514 
01515 - 1 Deb (the DBA).  Deb has global privileges on everything.  She needs
01516   them as she is the DBA.
01517 - 2 Pat (the PM).  Pat has the manager role globally, and is the project
01518   manager of project 102.  Pat can see all but the most confidential
01519   personal data, and all data about her project.
01520 - 3 Derick (the director).  Derick can see all personal and project
01521   data.  He is also the project manager for project 101, the secret
01522   project.
01523 - 4 Will (the worker).  Will has been assigned to both projects.  He has
01524   minimal privileges and cannot access project confidential data.
01525 - 5 Wilma (the worker).  Willma has been assigned to project 101.  She has
01526   minimal privileges and cannot access project confidential data.
01527 - 6 Fred (the fired DBA).  Fred has all of the privileges of Deb, except
01528   for can_connect privilege.  This prevents Fred from being able to do
01529   anything.
01530 
01531 Here is a sample session, showing the different access enjoyed by
01532 different users.
01533 
01534 \verbatim
01535 veildemo=> select connect_person(4);
01536  connect_person 
01537 ----------------
01538  t
01539 (1 row)
01540 
01541 veildemo=> select * from persons;
01542  person_id |    person_name    
01543 -----------+-------------------
01544          4 | Will (the worker)
01545 (1 row)
01546 
01547 veildemo=> select * from person_details;
01548  person_id | detail_type_id |    value     
01549 -----------+----------------+--------------
01550          4 |           1003 | 20050105
01551          4 |           1002 | Employee
01552          4 |           1004 | 30,000
01553          4 |           1005 | 19660102
01554          4 |           1006 | 123456789
01555          4 |           1007 | Subservience
01556 (6 rows)
01557 
01558 veildemo=> select * from project_details;
01559  project_id | detail_type_id |  value   
01560 ------------+----------------+----------
01561         102 |           1001 | 20050101
01562         102 |           1002 | Ongoing
01563 (2 rows)
01564 
01565 veildemo=> select connect_person(2);
01566  connect_person 
01567 ----------------
01568  t
01569 (1 row)
01570 
01571 veildemo=> select * from person_details;
01572  person_id | detail_type_id |       value       
01573 -----------+----------------+-------------------
01574          1 |           1003 | 20050102
01575          2 |           1003 | 20050103
01576          3 |           1003 | 20050104
01577          4 |           1003 | 20050105
01578          5 |           1003 | 20050106
01579          6 |           1003 | 20050107
01580          1 |           1002 | Employee
01581          2 |           1002 | Employee
01582          3 |           1002 | Employee
01583          4 |           1002 | Employee
01584          5 |           1002 | Employee
01585          6 |           1002 | Terminated
01586          2 |           1004 | 50,000
01587          1 |           1005 | 19610102
01588          2 |           1005 | 19600102
01589          3 |           1005 | 19650102
01590          4 |           1005 | 19660102
01591          5 |           1005 | 19670102
01592          2 |           1006 | 123456789
01593          1 |           1007 | Oracle, C, SQL
01594          2 |           1007 | Soft peoply-stuff
01595          3 |           1007 | None at all
01596          4 |           1007 | Subservience
01597          5 |           1007 | Subservience
01598 (24 rows)
01599 
01600 veildemo=> select * from project_details;
01601  project_id | detail_type_id |  value   
01602 ------------+----------------+----------
01603         102 |           1001 | 20050101
01604         102 |           1002 | Ongoing
01605         102 |           1008 | $100,000
01606 (3 rows)
01607 
01608 veildemo=>
01609 
01610 \endverbatim
01611 
01612 Next: \ref demo-code
01613 
01614 */
01615 /*! \page demo-code The Demo Code
01616 \section demo-codesec The Code
01617 \subsection demo-code-veil-init veil.veil_demo_init(performing_reset bool)
01618 
01619 This function is called at the start of each session, and whenever
01620 \ref API-control-reset is called.  The parameter, doing_reset, is
01621 false when called to initialise a session and true when called from
01622 veil_perform_reset().  It is registered with \ref API-control-init
01623 through the <code>veil.veil_demo_init_fns</code> table which is created
01624 as an inherited table of <code>veil.veil_init_fns</code>.  By
01625 registering the initialisation functions using a veil_demo-specific
01626 inherited table, when the veil_demo extension is dropped, so is the
01627 registration data for \ref demo-code-veil-init.
01628 
01629 \dontinclude veil_demo.sqs
01630 \skip veil_demo_init(doing
01631 \until init_reqd =
01632 
01633 The first task of veil_init() is to declare a set of Veil shared
01634 variables.  This is done by calling \ref API-variables-share.  This function
01635 returns true if the variable already exists, and creates the variable
01636 and returns false, if not.
01637 
01638 These variables are defined as shared because they will be identical for
01639 each session.  Making them shared means that only one session has to
01640 deal with the overhead of their initialisation.
01641 
01642 \dontinclude veil_demo.sqs
01643 \skip init_reqd =
01644 \until end if;
01645 
01646 We then check whether the shared variables must be initialised.  We will
01647 initialise them if they have not already been initialised by another
01648 session, or if we are performing a reset (see \ref API-control-reset).
01649 
01650 Each variable is initialised in its own way.  
01651 
01652 Ranges are set by a single call to \ref API-basic-init-range.  Ranges are
01653 used to create bitmap and array types of a suitable size.
01654 
01655 Int4Arrays are used to record mappings of one integer to another.  In
01656 the demo, they are used to record the mapping of detail_type_id to
01657 required_privilege_id.  We use this variable so that we can look-up the
01658 privilege required to access a given project_detail or person_detail
01659 without having to explicitly fetch from attribute_detail_types.
01660 
01661 Int4Arrays are initialised by a call to \ref API-intarray-init, and
01662 are populated by calling \ref API-intarray-set for each value to
01663 be recorded.  Note that rather than using a cursor to loop through each
01664 detail_type record, we use select count().  This requires less code and
01665 has the same effect.
01666 
01667 We use a BitmapArray to record the set of privileges for each role.  Its
01668 initialisation and population is handled in much the same way as
01669 described above for Int4Arrays, using the functions \ref
01670 API-bmarray-init and \ref API-bmarray-setbit.
01671 
01672 \until language
01673 
01674 The final section of code defines and initialises a set of session
01675 variables.  These are defined here to avoid getting undefined variable
01676 errors from any access function that may be called before an
01677 authenticated connection has been established.
01678 
01679 Note that this and all Veil related functions are defined with
01680 <code>security definer</code> attributes.  This means that the function
01681 will be executed with the privileges of the function's owner, rather
01682 than those of the invoker.  This is absolutely critical as the invoker
01683 must have no privileges on the base objects, or on the raw Veil
01684 functions themselves.  The only access to objects protected by Veil must
01685 be through user-defined functions and views.
01686 
01687 \subsection demo-code-connect-person connect_person(_person_id int4)
01688 
01689 This function is used to establish a connection from a specific person.
01690 In a real application this function would be provided with some form of
01691 authentication token for the user.  For the sake of simplicity the demo
01692 allows unauthenticated connection requests.
01693 
01694 \skip connect_person(_per
01695 \until language
01696 
01697 This function identifies the user, ensures that they have can_connect
01698 privilege.  It initialises the global_context bitmap to contain the
01699 union of all privileges for each role the person is assigned through
01700 person_roles.  It also sets up a bitmap hash containing a bitmap of
01701 privileges for each project to which the person is assigned.
01702 
01703 \subsection demo-code-global-priv i_have_global_priv(priv_id int4)
01704 
01705 This function is used to determine whether a user has a specified
01706 privilege in the global context.  It tests that the user is connected
01707 using \ref API-basic-int4-get, and then checks whether the
01708 specified privilege is present in the <code>global_context</code>
01709 bitmap.
01710 
01711 \skip function i_have_global_priv(priv
01712 \until security definer;
01713 
01714 The following example shows this function in use by the secured view,
01715 <code>privileges</code>:
01716 
01717 \skip create view privileges
01718 \until grant
01719 
01720 The privileges used above are <code>select_privileges</code> (10001),
01721 <code>insert_privileges</code> (10002), <code>update_privileges</code>
01722 (10003), and <code>delete_privileges</code> (10004).
01723 
01724 \subsection demo-code-personal-priv i_have_personal_priv(priv_id int4, person_id int4)
01725 
01726 This function determines whether a user has a specified privilege to a
01727 specified user's data, in the global or personal contexts.  It performs
01728 the same tests as for \ref demo-code-global-priv.  If the user
01729 does not have access in the global context, and the connected user is
01730 the same user as the owner of the data we are looking at, then we test
01731 whether the specified privilege exists in the <code>role_privs</code>
01732 bitmap array for the <code>Personal Context</code> role.
01733 
01734 \dontinclude veil_demo.sqs
01735 \skip function i_have_personal_priv(pr
01736 \until language
01737 
01738 Here is an example of this function in use from the persons secured view:
01739 
01740 \skip create view persons
01741 \until grant
01742 
01743 \subsection demo-code-project-priv i_have_project_priv(priv_id int4, project_id int4)
01744 This function determines whether a user has a specified privilege in the
01745 global or project contexts.  If the user does not have the global
01746 privilege, we check whether they have the privilege defined in the
01747 project_context BitmapHash.
01748 
01749 \dontinclude veil_demo.sqs
01750 \skip function i_have_project_priv(pr
01751 \until language
01752 
01753 Here is an example of this function in use from the instead-of insert
01754 trigger for the projects secured view:
01755 
01756 \skip create rule ii_projects 
01757 \until i_have_project_priv(10018, new.project_id);
01758 
01759 \subsection demo-code-proj-pers-priv i_have_proj_or_pers_priv(priv_id int4, project_id int4, person_id int4)
01760 This function checks all privileges.  It starts with the cheapest check
01761 first, and short-circuits as soon as a privilege is found.
01762 
01763 \dontinclude veil_demo.sqs
01764 \skip function i_have_proj_or_pers_priv(
01765 \until language
01766 
01767 Here is an example of this function in use from the instead-of update
01768 trigger for the assignments secured view:
01769 
01770 \skip create rule ii_assignments 
01771 \until i_have_proj_or_pers_priv(10027, old.project_id, old.person_id);
01772 
01773 \subsection demo-code-pers-detail-priv i_have_person_detail_priv(detail_id int4, person_id int4)
01774 This function is used to determine which types of person details are
01775 accessible to each user.  This provides distinct access controls to each
01776 attribute that may be recorded for a person.
01777 
01778 \dontinclude veil_demo.sqs
01779 \skip function i_have_person_detail_priv(
01780 \until language
01781 
01782 The function is shown in use, below, in the instead-of delete trigger
01783 for person_details.  Note that two distinct access functions are being
01784 used here.
01785 
01786 \skip create rule id_person_details
01787 \until i_have_person_detail_priv(old.detail_type_id, old.person_id);
01788 
01789 Next: \ref demo-uninstall
01790 
01791 */
01792 /*! \page demo-uninstall Removing The Demo Database
01793 \section demo-clean-up Removing The Demo Database
01794 In your veil_demo database execute:
01795 - <code>drop extension veil_demo;</code> 
01796 
01797 Next: \ref Management
01798 
01799 */
01800 /*! \page Management Managing Privileges, etc
01801 \section Management-sec Managing Privileges, etc
01802 The management of privileges and their assignments to roles, persons,
01803 etc are the key to securing a veil-based application.  It is therefore
01804 vital that privilege assignment is itself a privileged operation.
01805 
01806 The veil demo does not provide an example of how to do this, and this
01807 section does little more than raise the issue.
01808 
01809 IT IS VITAL THAT YOU CAREFULLY LIMIT HOW PRIVILEGES ARE MANIPULATED AND
01810 ASSIGNED!
01811 
01812 Here are some possible rules of thumb that you may wish to apply:
01813 
01814 - give only the most senior and trusted users the ability to assign
01815   privileges;
01816 - allow only the DBAs to create privileges;
01817 - allow only 1 or 2 security administrators to manage roles;
01818 - allow roles or privileges to be assigned only by users that have both
01819   the "assign_privileges"/"assign_roles" privileges, and that themselves
01820   have the privilege or role they are assigning;
01821 - consider having an admin privilege for each table and only allow users
01822   to assign privileges on X if they have "admin_x" privilege;
01823 - limit the users who have access to the role/privilege management
01824   functions, and use function-level privileges to enforce this;
01825 - audit/log all assignments of privileges and roles;
01826 - send email to the security administrator whenever role_privileges are
01827   manipulated and when roles granting high-level privileges are granted.
01828 
01829 Next: \ref Esoteria
01830 
01831 */
01832 /*! \page Esoteria Exotic and Esoteric uses of Veil
01833 
01834 \section Esoteria-sec Exotic and Esoteric uses of Veil
01835 Actually this is neither exotic nor particularly esoteric.  The title is
01836 simply wishful thinking on the author's part.
01837 \subsection layered-sessions Multi-Layered Connections
01838 So far we have considered access controls based only on the user.  If we
01839 wish to be more paranoid, and perhaps we should, we may also consider
01840 limiting the access rights of each application.
01841 
01842 This might mean that reporting applications would have no ability to
01843 update data, that financial applications would have no access to
01844 personnel data, and that personnel apps would have no access to business
01845 data.
01846 
01847 This can be done in addition to the user-level checks, so that even if I
01848 have DBA privilege, I can not subvert the personnel reporting tools to
01849 modify financial data.
01850 
01851 All access functions would check the service's privileges in addition to
01852 the user's before allowing any operation.
01853 
01854 This could be implemented with a connect_service() function that would
01855 be called only once per session and that *must* be called prior to
01856 connecting any users.  Alternatively, the connected service could be
01857 inferred from the account to which the service is connected.
01858 
01859 \subsection columns Column-Level Access Controls
01860 
01861 Although veil is primarily intended for row-based access controls,
01862 column-based is also possible.  If this is required it may be better to
01863 use a highly normalised data model where columns are converted instead
01864 into attributes, much like the person_details and project_details tables
01865 from the demo application (\ref demo-sec).
01866 
01867 If this is not possible then defining access_views that only show
01868 certain columns can be done something like this:
01869 
01870 \verbatim
01871 create view wibble(key, col1, col2, col3) as
01872 select key, 
01873        case when have_col_priv(100001) then col1 else null end,
01874        case when have_col_priv(100002) then col2 else null end,
01875        case when have_col_priv(100003) then col3 else null end
01876 where  have_row_priv(1000);
01877 \endverbatim
01878 
01879 The instead-of triggers for this are left as an exercise.
01880 
01881 Next: \ref install
01882 
01883 */
01884 /*! \page install Installation and Configuration
01885 \section install_sec Installation
01886 \subsection Get Getting Veil
01887 Veil can be downloaded as a gzipped tarball from
01888 http://pgfoundry.org/projects/veil/
01889 
01890 Since version 9.1, git is no longer available from cvs on pgfoundry.
01891 Development has switched to using git.  The primary git repository is 
01892 git://github.com/marcmunro/veil.git
01893 
01894 To checkout from git, create a suitable directory and do:
01895 
01896 \verbatim
01897  git clone git://github.com/marcmunro/veil.git
01898 \endverbatim
01899 
01900 An alternative repository is also available here:
01901 git@bloodnok.com:veil.git
01902 
01903 \subsection Pre-requisites Pre-requisites
01904 You must have a copy of the Postgresql header files available in order
01905 to build Veil.  For this, you may need to install the postgres developer
01906 packages for your OS, or even build postgres from source.
01907 \subsection build-sub Building Veil
01908 Veil is built using the postgresql extension building infastructure,
01909 PGXS.  In order to be able to build using PGXS, the following command
01910 must return the location of the PGXS makefile:
01911 \verbatim
01912 $ pg_config --pgxs
01913 \endverbatim
01914 You may need to modify your PATH variable to make pg_config usable, or
01915 ensure it is for the correct postgresql version.
01916 
01917 To build the postgres extensions, simply run make with no target:
01918 \verbatim
01919 $ make
01920 \endverbatim
01921 
01922 To build the veil documentation (the documentation you are now reading)
01923 use make docs.
01924 
01925 Note that the build system deliberately avoids using make recursively.
01926 Search the Web for "Recursive Make Considered Harmful" for the reasons
01927 why.  This makes the construction of the build system a little different
01928 from what you may be used to.  This may or may not turn out to be a good
01929 thing.  \ref Feedback "Feedback" is welcomed.
01930 
01931 \subsection Install Installing Veil
01932 As the postgres, pgsql, or root user, run <code>make install</code>.
01933 \verbatim
01934 make install
01935 \endverbatim
01936 
01937 \section configuration Configuration
01938 To configure Veil, the following lines should be added to your
01939 postgresql.conf:
01940 \code
01941 shared_preload_libraries = '<path to shared library>/veil.so' 
01942 
01943 custom_variable_classes = 'veil'
01944 
01945 #veil.dbs_in_cluster = 1
01946 #veil.shared_hash_elems = 32
01947 #veil.shmem_context_size = 16384
01948 \endcode
01949 
01950 The three configuration options, commented out above, are:
01951 - dbs_in_cluster
01952   The number of databases, within the database cluster, that
01953   will use Veil.  Each such database will be allocated 2 chunks of
01954   shared memory (of shmem_context_size), and a single LWLock.
01955   It defaults to 1.
01956 
01957 - shared_hash_elems
01958   This describes how large a hash table should be created for veil
01959   shared variables.  It defaults to 32.  If you have more than about 20
01960   shared variables you may want to increase this to improve
01961   performance.  This setting does not limit the number of variables that
01962   may be defined, it just limits how efficiently they may be accessed.
01963 
01964 - shmem_context_size
01965   This sets an upper limit on the amount of shared memory for a single
01966   Veil shared memory context (there will be two of these).  It defaults 
01967   to 16K.  Increase this if you have many shared memory structures.
01968 
01969 \subsection Regression Regression Tests
01970 Veil comes with a built-in regression test suite.  Use <code>make
01971 regress</code> or <code>make check</code> (after installing and
01972 configuring Veil) to run this.  You will need superuser access to
01973 Postgres in order to create the regression test database.  The
01974 regression test assumes you will have a postgres superuser account named
01975 the same as your OS account.  If pg_hba.conf disallows "trust"ed access
01976 locally, then you will need to provide a password for this account in
01977 your .pgpass file (see postgres documentation for details).
01978 
01979 The regression tests are all contained within the regress directory and
01980 are run by the regress.sh shell script.  Use the -h option to get
01981 fairly detailed help.
01982 
01983 \subsection Debugging Debugging
01984 If you encounter problems with Veil, you may want to try building with
01985 debug enabled.  Define the variable VEIL_DEBUG on the make command line
01986 to add extra debug code to the executable:
01987 \verbatim
01988 $ make clean; make VEIL_DEBUG=1 all
01989 \endverbatim
01990 
01991 This is a transient feature and not as pervasive as it could be.  If you
01992 need help with debugging please contact the author.
01993 
01994 Next: \ref History
01995 
01996 */
01997 /*! \page History History and Compatibility
01998 \section past Changes History
01999 \subsection v1_0 Version 9.1.0 (Stable) (2011-07-22)
02000 This is the first version of Veil to be considered production ready and
02001 completely stable.  It is for use only with PostgreSQL 9.1.  Support for
02002 older versions of PostgreSQL has been removed in this version.
02003 
02004 Major changes include:
02005 - revamp of the build system to use PGXS and the new PostgreSQL 9.1
02006   extensions mechanism.  Veil is now built as an extension.
02007 - modification to the veil_init() mechanism, allowing custom
02008   initialisation functions to be registered through the table
02009   veil.veil_init_fns
02010 - removal of the old veil_trial mechanism, which allowed Veil to be
02011   tried out without fully installing it.  This has removed much
02012   unnecessary complexity.
02013 - much general code cleanup, including the removal of conditional code
02014   for older PostgreSQL versions.
02015 - documentation changes, including improved comments for Veil
02016   functions.
02017 
02018 \subsection v9_12 Version 0.9.12 (2010-11-19)
02019 Release for compatibility with PostgreSQL V9.0.  Minor bugfixes and
02020 improvements to the build system.  Also added documentation about Veil's
02021 limitations.
02022 
02023 \subsection v9_11 Version 0.9.11 (2010-03-12)
02024 Bugfix release, fixing a serious memory corruption bug that has existed
02025 in all previous versions.  Users are strongly encouraged to avoid using
02026 older versions of Veil.
02027 
02028 The version number has been deliberatley bumped past 0.9.10 to emphasize
02029 that the last part of the version is a two digit number.
02030 
02031 \subsection v9_9 Version 0.9.9 (2009-07-06)
02032 New release to coincide with PostgreSQL V8.4.
02033 
02034 \subsection v9_8 Version 0.9.8 (2008-02-06)
02035 This is the first Beta release.  It incorporates a few bug fixes, a new
02036 serialisation API, improvements to the autoconf setup and makefiles, and
02037 some documentation improvements.  The status of Veil has been raised to
02038 Beta in recognition of its relative stability.
02039 
02040 \subsection v9_6 Version 0.9.6 (2008-02-06)
02041 This release has minor changes to support PostgreSQL 8.3.
02042 
02043 \subsection v9_5 Version 0.9.5 (2007-07-31)
02044 This is a bugifx release, fixing a memory allocation bug in the use of 
02045 bitmap_refs.  There are also fixes for minor typos, etc.
02046 
02047 \subsection v9_4 Version 0.9.4 (2007-02-21)
02048 This is a bugifx release, providing:
02049  - fix for major bug with recursive handling of spi connect, etc;
02050  - improvement to session initialisation code to do more up-front work
02051    in ensure_init();
02052  - safer initialisation of malloc'd data structures;
02053  - improved error messages for shared memory exhaustion cases;
02054  - addition of debug code including canaries in data structures;
02055  - improvement to autoconf to better support Debian GNU/Linux, and OSX;
02056  - improvement to autoconf/make for handling paths containing spaces;
02057  - improvement to regression tests to better support OSX;
02058  - removal of spurious debug warning messages.
02059 
02060 \subsection v9_3 Version 0.9.3 (2006-10-31) 
02061 This version uses the new Postgres API for reserving shared memory for
02062 add-ins.  It also allows the number of Veil-enabled databases for a
02063 cluster to be configured, and refactors much of the shared memory code.
02064 A small fix for the Darwin makefile was also made.
02065 
02066 \subsection v9_2 Version 0.9.2 (2006-10-01) 
02067 This version was released to coincide with Postgres 8.2beta1 and first
02068 made use of new Postgres APIs to allow Veil to be a good Postgres
02069 citizen.  
02070 
02071 With prior versions of Veil, or prior versions of Postgres, Veil steals
02072 from Postgres the shared memory that it requires.  This can lead to the
02073 exhaustion of Postgres shared memory.
02074 
02075 Unfortunately, the Postgres API for shared memory reservation had to
02076 change follwing 8.2.beta1, and this version of Veil is therefore deprecated.
02077 
02078 \subsection v9_1 Version 0.9.1 (2006-07-04)
02079 This release fixed a small number of bugs and deficiencies:
02080 - major error in veil_perform_reset that prevented proper use of the two
02081 interdependant shared memory contexts
02082 - minor improvements in the build process to "configure" and friends
02083 - minor documentation improvements
02084 
02085 \subsection v9_0 Version 0.9.0  (2005-10-04) 
02086 This was the first public alpha release of Veil.
02087 
02088 \section forecast Change Forecast
02089 New versions will be released with each new major version of
02090 PostgreSQL.  Once there are three PostgreSQL versions for which Veil has
02091 been at production status, the change history and support matrix for for
02092 pre-production versions will be removed from this documentation.
02093 
02094 \section compatibility Supported versions of Postgres
02095 <TABLE>
02096   <TR>
02097     <TD rowspan=2>Veil version</TD>
02098     <TD colspan=9>Postgres Version</TD>
02099   </TR>
02100   <TR>
02101     <TD>7.4</TD>
02102     <TD>8.0</TD>
02103     <TD>8.1</TD>
02104     <TD>8.2beta1</TD>
02105     <TD>8.2</TD>
02106     <TD>8.3</TD>
02107     <TD>8.4</TD>
02108     <TD>9.0</TD>
02109     <TD>9.1</TD>
02110   </TR>
02111   <TR>
02112     <TD>0.9.0 Alpha</TD>
02113     <TD>1</TD>
02114     <TD>1</TD>
02115     <TD>1</TD>
02116     <TD>-</TD>
02117     <TD>-</TD>
02118     <TD>-</TD>
02119     <TD>-</TD>
02120     <TD>-</TD>
02121     <TD>-</TD>
02122   </TR>
02123   <TR>
02124     <TD>0.9.1 Alpha</TD>
02125     <TD>1</TD>
02126     <TD>1</TD>
02127     <TD>1</TD>
02128     <TD>-</TD>
02129     <TD>-</TD>
02130     <TD>-</TD>
02131     <TD>-</TD>
02132     <TD>-</TD>
02133     <TD>-</TD>
02134   </TR>
02135   <TR>
02136     <TD>0.9.2 Alpha</TD>
02137     <TD>-</TD>
02138     <TD>1</TD>
02139     <TD>1</TD>
02140     <TD>2</TD>
02141     <TD>-</TD>
02142     <TD>-</TD>
02143     <TD>-</TD>
02144     <TD>-</TD>
02145     <TD>-</TD>
02146   </TR>
02147   <TR>
02148     <TD>0.9.3 Alpha</TD>
02149     <TD>-</TD>
02150     <TD>1</TD>
02151     <TD>1</TD>
02152     <TD>-</TD>
02153     <TD>3</TD>
02154     <TD>-</TD>
02155     <TD>-</TD>
02156     <TD>-</TD>
02157     <TD>-</TD>
02158   </TR>
02159   <TR>
02160     <TD>0.9.4 Alpha</TD>
02161     <TD>-</TD>
02162     <TD>1</TD>
02163     <TD>1</TD>
02164     <TD>-</TD>
02165     <TD>3</TD>
02166     <TD>-</TD>
02167     <TD>-</TD>
02168     <TD>-</TD>
02169     <TD>-</TD>
02170   </TR>
02171   <TR>
02172     <TD>0.9.5 Alpha</TD>
02173     <TD>-</TD>
02174     <TD>1</TD>
02175     <TD>1</TD>
02176     <TD>-</TD>
02177     <TD>3</TD>
02178     <TD>-</TD>
02179     <TD>-</TD>
02180     <TD>-</TD>
02181     <TD>-</TD>
02182   </TR>
02183   <TR>
02184     <TD>0.9.6 Alpha</TD>
02185     <TD>-</TD>
02186     <TD>1</TD>
02187     <TD>1</TD>
02188     <TD>-</TD>
02189     <TD>3</TD>
02190     <TD>3</TD>
02191     <TD>-</TD>
02192     <TD>-</TD>
02193     <TD>-</TD>
02194   </TR>
02195   <TR>
02196     <TD>0.9.8 Beta</TD>
02197     <TD>-</TD>
02198     <TD>1</TD>
02199     <TD>1</TD>
02200     <TD>-</TD>
02201     <TD>3</TD>
02202     <TD>3</TD>
02203     <TD>-</TD>
02204     <TD>-</TD>
02205     <TD>-</TD>
02206   </TR>
02207   <TR>
02208     <TD>0.9.9 Beta</TD>
02209     <TD>-</TD>
02210     <TD>1</TD>
02211     <TD>1</TD>
02212     <TD>-</TD>
02213     <TD>3</TD>
02214     <TD>3</TD>
02215     <TD>3</TD>
02216     <TD>-</TD>
02217     <TD>-</TD>
02218   </TR>
02219   <TR>
02220     <TD>0.9.11 Beta</TD>
02221     <TD>-</TD>
02222     <TD>1</TD>
02223     <TD>1</TD>
02224     <TD>-</TD>
02225     <TD>3</TD>
02226     <TD>3</TD>
02227     <TD>3</TD>
02228     <TD>-</TD>
02229     <TD>-</TD>
02230   </TR>
02231   <TR>
02232     <TD>0.9.12 Beta</TD>
02233     <TD>-</TD>
02234     <TD>-</TD>
02235     <TD>-</TD>
02236     <TD>-</TD>
02237     <TD>-</TD>
02238     <TD>3</TD>
02239     <TD>3</TD>
02240     <TD>3</TD>
02241     <TD>-</TD>
02242   </TR>
02243   <TR>
02244     <TD>9.1.0 (Stable)</TD>
02245     <TD>-</TD>
02246     <TD>-</TD>
02247     <TD>-</TD>
02248     <TD>-</TD>
02249     <TD>-</TD>
02250     <TD>-</TD>
02251     <TD>-</TD>
02252     <TD>-</TD>
02253     <TD>Yes</TD>
02254   </TR>
02255 </TABLE>
02256 Notes:
02257 
02258 1) These combinations of Veil and Postgres provide no configuration
02259    options for shared memory.  Veil's shared memory may be exhausted by
02260    too many requests for large shared objects.  Furthermore, Postgres'
02261    own shared memory may be easily exhausted by creating too many
02262    Veil-using databases within a cluster.
02263 
02264 2) This version is deprecated
02265 
02266 3) These combinations of Veil and Postgres provide full configuration
02267    options for shared memory usage, and Veil cooperates with Postgres
02268    for the allocation of such memory meaning that it is not possible to
02269    use Veil to exhaust Postgres' shared memory.  This is the minimum
02270    Veil configuration recommended for production use.
02271 
02272 \section platforms Supported Platforms
02273 Veil should be buildable on any platform supported by PostgreSQL and
02274 PGXS. 
02275 
02276 Next: \ref Feedback
02277 
02278 */
02279 /*! \page Feedback Bugs and Feedback
02280 \section Feedback Bugs and Feedback
02281 For general feedback, to start and follow discussions, etc please join
02282 the veil-general@pgfoundry.org mailing list.
02283 
02284 If you wish to report a bug or request a feature, please send mail to
02285 veil-general@pgfoundry.org
02286 
02287 If you encounter a reproducible veil bug that causes a database server
02288 crash, a gdb backtrace would be much appreciated.  To generate a
02289 backtrace, you will need to login to the postgres owner account on the
02290 database server.  Then identify the postgres backend process associated
02291 with the database session that is going to crash.  The following command
02292 identifies the backend pid for a database session started by marc:
02293 
02294 \verbatim
02295 $ ps auwwx | grep ^postgres.*ma[r]c | awk '{print $2}'
02296 \endverbatim
02297 
02298 Now you invoke gdb with the path to the postgres binary and the pid for
02299 the backend, eg:
02300 
02301 \verbatim
02302 $ gdb /usr/lib/postgresql/9.1/bin/postgres 5444
02303 \endverbatim
02304 
02305 Hit c and Enter to get gdb to allow the session to continue.   Now,
02306 reproduce the crash from your database session.  When the crash occurs,
02307 your gdb session will return to you.  Now type bt and Enter to get a
02308 backtrace. 
02309 
02310 If you wish to contact the author offlist, you can find him at his website
02311 http://bloodnok.com/Marc.Munro
02312 
02313 Next: \ref Performance
02314 
02315 */
02316 /*! \page Performance Performance
02317 \section perf Performance
02318 Attempts to benchmark veil using pgbench have not been very successful.
02319 It seems that the overhead of veil is small enough that it is
02320 overshadowed by the level of "noise" within pgbench.
02321 
02322 Based on this inability to properly benchmark veil, the author is going
02323 to claim that "it performs well".
02324 
02325 To put this into perspective, if your access functions do not require
02326 extra fetches to be performed in order to establish your access rights,
02327 you are unlikely to notice or be able to measure any performance hit
02328 from veil.
02329 
02330 If anyone can provide good statistical evidence of a performance hit,
02331 the author would be most pleased to hear from you.
02332 
02333 Next: \ref Credits
02334 
02335 */
02336 /*! \page Credits Credits
02337 \section Credits
02338 The Entity Relationship Diagram in section \ref demo-erd was produced
02339 automatically from an XML definition of the demo tables, using Autograph
02340 from CF Consulting.  Thanks to Colin Fox for allowing its use.
02341 
02342 Thanks to the PostgreSQL core team for providing PostgreSQL.
02343 
02344 Thanks to pgfoundry for providing a home for this project.
02345 */
02346 

Generated on Mon Sep 12 15:26:45 2011 for Veil by  doxygen 1.5.6