text:
- Prev: Connection Pooling
- Next: Column and Data Types
- Table of Contents | Index | view source
Schema Definition Language
Table of Contents
- Schema Definition Language
Previous Topic
Next Topic
Project Versions
Quick Search
Schema Definition Language¶
Describing Databases with MetaData¶
The core of SQLAlchemy's query and object mapping operations are supported by database metadata, which is comprised of Python objects that describe tables and other schema-level objects. These objects are at the core of three major types of operations - issuing CREATE and DROP statements (known as DDL), constructing SQL queries, and expressing information about structures that already exist within the database.
Database metadata can be expressed by explicitly naming the various components and their properties, using constructs such as Table, Column, ForeignKey and Sequence, all of which are imported from the sqlalchemy.schema package. It can also be generated by SQLAlchemy using a process called reflection, which means you start with a single object such as Table, assign it a name, and then instruct SQLAlchemy to load all the additional information related to that name from a particular engine source.
A key feature of SQLAlchemy's database metadata constructs is that they are designed to be used in a declarative style which closely resembles that of real DDL. They are therefore most intuitive to those who have some background in creating real schema generation scripts.
A collection of metadata entities is stored in an object aptly named MetaData:
from sqlalchemy import *
metadata = MetaData()
MetaData is a container object that keeps together many different features of a database (or multiple databases) being described.
To represent a table, use the Table class. Its two primary arguments are the table name, then the MetaData object which it will be associated with. The remaining positional arguments are mostly Column objects describing each column:
user = Table('user', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('email_address', String(60)),
Column('password', String(20), nullable = False)
Above, a table called user is described, which contains four columns. The primary key of the table consists of the user_id column. Multiple columns may be assigned the primary_key=True flag which denotes a multi-column primary key, known as a composite primary key.
Note also that each column describes its datatype using objects corresponding to genericized types, such as Integer and String. SQLAlchemy features dozens of types of varying levels of specificity as well as the ability to create custom types. Documentation on the type system can be found at types.
Accessing Tables and Columns¶
The MetaData object contains all of the schema constructs we've associated with it. It supports a few methods of accessing these table objects, such as the sorted_tables accessor which returns a list of each Table object in order of foreign key dependency (that is, each table is preceded by all tables which it references):
>>> for t in metadata.sorted_tables:
... print t.name
In most cases, individual Table objects have been explicitly declared, and these objects are typically accessed directly as module-level variables in an application. Once a Table has been defined, it has a full set of accessors which allow inspection of its properties. Given the following Table definition:
employees = Table('employees', metadata,
Column('employee_id', Integer, primary_key=True),
Column('employee_name', String(60), nullable=False),
Column('employee_dept', Integer, ForeignKey("departments.department_id"))
Note the ForeignKey object used in this table - this construct defines a reference to a remote table, and is fully described in Defining Foreign Keys. Methods of accessing information about this table include:
# access the column "EMPLOYEE_ID":
# or just
# via string
# iterate through all columns
for c in employees.c:
print c
# get the table's primary key columns
for primary_key in employees.primary_key:
print primary_key
# get the table's foreign key objects:
for fkey in employees.foreign_keys:
print fkey
# access the table's MetaData:
# access the table's bound Engine or Connection, if its MetaData is bound:
# access a column's name, type, nullable, primary key, foreign key
# get the "key" of a column, which defaults to its name, but can
# be any user-defined string:
# access a column's table:
employees.c.employee_id.table is employees
# get the table related by a foreign key
Creating and Dropping Database Tables¶
Once you've defined some Table objects, assuming you're working with a brand new database one thing you might want to do is issue CREATE statements for those tables and their related constructs (as an aside, it's also quite possible that you don't want to do this, if you already have some preferred methodology such as tools included with your database or an existing scripting system - if that's the case, feel free to skip this section - SQLAlchemy has no requirement that it be used to create your tables).
The usual way to issue CREATE is to use create_all() on the MetaData object. This method will issue queries that first check for the existence of each individual table, and if not found will issue the CREATE statements:
engine = create_engine('sqlite:///:memory:') metadata = MetaData() user = Table('user', metadata, Column('user_id', Integer, primary_key = True), Column('user_name', String(16), nullable = False), Column('email_address', String(60), key='email'), Column('password', String(20), nullable = False) ) user_prefs = Table('user_prefs', metadata, Column('pref_id', Integer, primary_key=True), Column('user_id', Integer, ForeignKey("user.user_id"), nullable=False), Column('pref_name', String(40), nullable=False), Column('pref_value', String(100)) ) sqlmetadata.create_all(engine)PRAGMA table_info(user){} CREATE TABLE user( user_id INTEGER NOT NULL PRIMARY KEY, user_name VARCHAR(16) NOT NULL, email_address VARCHAR(60), password VARCHAR(20) NOT NULL ) PRAGMA table_info(user_prefs){} CREATE TABLE user_prefs( pref_id INTEGER NOT NULL PRIMARY KEY, user_id INTEGER NOT NULL REFERENCES user(user_id), pref_name VARCHAR(40) NOT NULL, pref_value VARCHAR(100) )
create_all() creates foreign key constraints between tables usually inline with the table definition itself, and for this reason it also generates the tables in order of their dependency. There are options to change this behavior such that ALTER TABLE is used instead.
Dropping all tables is similarly achieved using the drop_all() method. This method does the exact opposite of create_all() - the presence of each table is checked first, and tables are dropped in reverse order of dependency.
Creating and dropping individual tables can be done via the create() and drop() methods of Table. These methods by default issue the CREATE or DROP regardless of the table being present:
engine = create_engine('sqlite:///:memory:')
meta = MetaData()
employees = Table('employees', meta,
Column('employee_id', Integer, primary_key=True),
Column('employee_name', String(60), nullable=False, key='name'),
Column('employee_dept', Integer, ForeignKey("departments.department_id"))
CREATE TABLE employees(
employee_id SERIAL NOT NULL PRIMARY KEY,
employee_name VARCHAR(60) NOT NULL,
employee_dept INTEGER REFERENCES departments(department_id)
drop() method:
DROP TABLE employee
To enable the "check first for the table existing" logic, add the checkfirst=True argument to create() or drop():
employees.create(engine, checkfirst=True)
employees.drop(engine, checkfirst=False)
Altering Schemas through Migrations¶
While SQLAlchemy directly supports emitting CREATE and DROP statements for schema constructs, the ability to alter those constructs, usually via the ALTER statement as well as other database-specific constructs, is outside of the scope of SQLAlchemy itself. While it's easy enough to emit ALTER statements and similar by hand, such as by passing a string to Connection.execute() or by using the DDL construct, it's a common practice to automate the maintenance of database schemas in relation to application code using schema migration tools.
There are two major migration tools available for SQLAlchemy:
- Alembic - Written by the author of SQLAlchemy, Alembic features a highly customizable environment and a minimalistic usage pattern, supporting such features as transactional DDL, automatic generation of "candidate" migrations, an "offline" mode which generates SQL scripts, and support for branch resolution.
- SQLAlchemy-Migrate - The original migration tool for SQLAlchemy, SQLAlchemy-Migrate is widely used and continues under active development. SQLAlchemy-Migrate includes features such as SQL script generation, ORM class generation, ORM model comparison, and extensive support for SQLite migrations.
Specifying the Schema Name¶
Some databases support the concept of multiple schemas. A Table can reference this by specifying the schema keyword argument:
financial_info = Table('financial_info', meta,
Column('id', Integer, primary_key=True),
Column('value', String(100), nullable=False),
Within the MetaData collection, this table will be identified by the combination of financial_info and remote_banks. If another table called financial_info is referenced without the remote_banks schema, it will refer to a different Table. ForeignKey objects can specify references to columns in this table using the form remote_banks.financial_info.id.
The schema argument should be used for any name qualifiers required, including Oracle's "owner" attribute and similar. It also can accommodate a dotted name for longer schemes:
Backend-Specific Options¶
Table supports database-specific options. For example, MySQL has different table backend types, including "MyISAM" and "InnoDB". This can be expressed with Table using mysql_engine:
addresses = Table('engine_email_addresses', meta,
Column('address_id', Integer, primary_key = True),
Column('remote_user_id', Integer, ForeignKey(users.c.user_id)),
Column('email_address', String(20)),
Other backends may support table-level options as well - these would be described in the individual documentation sections for each dialect.
Column, Table, MetaData API¶
- class sqlalchemy.schema.Column(*args, **kwargs)¶
Bases: sqlalchemy.schema.SchemaItem, sqlalchemy.sql.expression.ColumnClause
Represents a column in a database table.
- __eq__(other)¶
- inherited from the __eq__() method of ColumnOperators
Implement the == operator.
In a column context, produces the clause a = b. If the target is None, produces a IS NULL.
- __init__(*args, **kwargs)¶
Construct a new Column object.
Parameters: - name –
The name of this column as represented in the database. This argument may be the first positional argument, or specified via keyword.
Names which contain no upper case characters will be treated as case insensitive names, and will not be quoted unless they are a reserved word. Names with any number of upper case characters will be quoted and sent exactly. Note that this behavior applies even for databases which standardize upper case names as case insensitive such as Oracle.
The name field may be omitted at construction time and applied later, at any time before the Column is associated with a Table. This is to support convenient usage within the declarative extension.
- type_ –
The column's type, indicated using an instance which subclasses TypeEngine. If no arguments are required for the type, the class of the type can be sent as well, e.g.:
# use a type with arguments Column('data', String(50)) # use no arguments Column('level', Integer)
The type argument may be the second positional argument or specified by keyword.
There is partial support for automatic detection of the type based on that of a ForeignKey associated with this column, if the type is specified as None. However, this feature is not fully implemented and may not function in all cases.
- *args – Additional positional arguments include various SchemaItem derived constructs which will be applied as options to the column. These include instances of Constraint, ForeignKey, ColumnDefault, and Sequence. In some cases an equivalent keyword argument is available such as server_default, default and unique.
- autoincrement –
This flag may be set to False to indicate an integer primary key column that should not be considered to be the "autoincrement" column, that is the integer primary key column which generates values implicitly upon INSERT and whose value is usually returned via the DBAPI cursor.lastrowid attribute. It defaults to True to satisfy the common use case of a table with a single integer primary key column. If the table has a composite primary key consisting of more than one integer column, set this flag to True only on the column that should be considered "autoincrement".
The setting only has an effect for columns which are:
- Integer derived (i.e. INT, SMALLINT, BIGINT).
- Part of the primary key
- Are not referenced by any foreign keys, unless
the value is specified as 'ignore_fk'
New in version 0.7.4.
- have no server side or client side defaults (with the exception of Postgresql SERIAL).
The setting has these two effects on columns that meet the above criteria:
- DDL issued for the column will include database-specific keywords intended to signify this column as an "autoincrement" column, such as AUTO INCREMENT on MySQL, SERIAL on Postgresql, and IDENTITY on MS-SQL. It does not issue AUTOINCREMENT for SQLite since this is a special SQLite flag that is not required for autoincrementing behavior. See the SQLite dialect documentation for information on SQLite's AUTOINCREMENT.
- The column will be considered to be available as cursor.lastrowid or equivalent, for those dialects which "post fetch" newly inserted identifiers after a row has been inserted (SQLite, MySQL, MS-SQL). It does not have any effect in this regard for databases that use sequences to generate primary key identifiers (i.e. Firebird, Postgresql, Oracle).
Changed in version 0.7.4: autoincrement accepts a special value 'ignore_fk' to indicate that autoincrementing status regardless of foreign key references. This applies to certain composite foreign key setups, such as the one demonstrated in the ORM documentation at Rows that point to themselves / Mutually Dependent Rows.
- default –
A scalar, Python callable, or ColumnElement expression representing the default value for this column, which will be invoked upon insert if this column is otherwise not specified in the VALUES clause of the insert. This is a shortcut to using ColumnDefault as a positional argument; see that class for full detail on the structure of the argument.
Contrast this argument to server_default which creates a default generator on the database side.
- doc – optional String that can be used by the ORM or similar to document attributes. This attribute does not render SQL comments (a future attribute 'comment' will achieve that).
- key – An optional string identifier which will identify this Column object on the Table. When a key is provided, this is the only identifier referencing the Column within the application, including ORM attribute mapping; the name field is used only when rendering SQL.
- index – When True, indicates that the column is indexed. This is a shortcut for using a Index construct on the table. To specify indexes with explicit names or indexes that contain multiple columns, use the Index construct instead.
- info – Optional data dictionary which will be populated into the SchemaItem.info attribute of this object.
- nullable – If set to the default of True, indicates the column will be rendered as allowing NULL, else it's rendered as NOT NULL. This parameter is only used when issuing CREATE TABLE statements.
- onupdate – A scalar, Python callable, or ClauseElement representing a default value to be applied to the column within UPDATE statements, which wil be invoked upon update if this column is not present in the SET clause of the update. This is a shortcut to using ColumnDefault as a positional argument with for_update=True.
- primary_key – If True, marks this column as a primary key column. Multiple columns can have this flag set to specify composite primary keys. As an alternative, the primary key of a Table can be specified via an explicit PrimaryKeyConstraint object.
- server_default –
A FetchedValue instance, str, Unicode or text() construct representing the DDL DEFAULT value for the column.
String types will be emitted as-is, surrounded by single quotes:
Column('x', Text, server_default="val") x TEXT DEFAULT 'val'
A text() expression will be rendered as-is, without quotes:
Column('y', DateTime, server_default=text('NOW()'))0 y DATETIME DEFAULT NOW()
Strings and text() will be converted into a DefaultClause object upon initialization.
Use FetchedValue to indicate that an already-existing column will generate a default value on the database side which will be available to SQLAlchemy for post-fetch after inserts. This construct does not specify any DDL and the implementation is left to the database, such as via a trigger.
- server_onupdate – A FetchedValue instance representing a database-side default generation function. This indicates to SQLAlchemy that a newly generated value will be available after updates. This construct does not specify any DDL and the implementation is left to the database, such as via a trigger.
- quote – Force quoting of this column's name on or off, corresponding to True or False. When left at its default of None, the column identifier will be quoted according to whether the name is case sensitive (identifiers with at least one upper case character are treated as case sensitive), or if it's a reserved word. This flag is only needed to force quoting of a reserved word which is not known by the SQLAlchemy dialect.
- unique – When True, indicates that this column contains a unique constraint, or if index is True as well, indicates that the Index should be created with the unique flag. To specify multiple columns in the constraint/index or to specify an explicit name, use the UniqueConstraint or Index constructs explicitly.
- name –
- __le__(other)¶
- inherited from the __le__() method of ColumnOperators
Implement the <= operator.
In a column context, produces the clause a <= b.
- __lt__(other)¶
- inherited from the __lt__() method of ColumnOperators
Implement the < operator.
In a column context, produces the clause a < b.
- __ne__(other)¶
- inherited from the __ne__() method of ColumnOperators
Implement the != operator.
In a column context, produces the clause a != b. If the target is None, produces a IS NOT NULL.
- anon_label¶
- inherited from the anon_label attribute of ColumnElement
provides a constant 'anonymous label' for this ColumnElement.
This is a label() expression which will be named at compile time. The same label() is returned each time anon_label is called so that expressions can reference anon_label multiple times, producing the same label name at compile time.
the compiler uses this function automatically at compile time for expressions that are known to be 'unnamed' like binary expressions and function calls.
- append_foreign_key(fk)¶
- asc()¶
- inherited from the asc() method of ColumnOperators
Produce a asc() clause against the parent object.
- base_columns¶
- inherited from the base_columns attribute of ColumnElement
- between(cleft, cright)¶
- inherited from the between() method of ColumnOperators
Produce a between() clause against the parent object, given the lower and upper range.
- bind = None¶
- collate(collation)¶
- inherited from the collate() method of ColumnOperators
Produce a collate() clause against the parent object, given the collation string.
- comparator¶
- inherited from the comparator attribute of ColumnElement
- compare(other, use_proxies=False, equivalents=None, **kw)¶
- inherited from the compare() method of ColumnElement
Compare this ColumnElement to another.
Special arguments understood:
Parameters: - use_proxies – when True, consider two columns that share a common base column as equivalent (i.e. shares_lineage())
- equivalents – a dictionary of columns as keys mapped to sets of columns. If the given "other" column is present in this dictionary, if any of the columns in the corresponding set() pass the comparison test, the result is True. This is used to expand the comparison to other columns that may be known to be equivalent to this one via foreign key or other criterion.
- compile(bind=None, dialect=None, **kw)¶
- inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield a string representation of the result. The Compiled object also can return a dictionary of bind parameter names and values using the params accessor.
Parameters: - bind – An Engine or Connection from which a Compiled will be acquired. This argument takes precedence over this ClauseElement's bound engine, if any.
- column_keys – Used for INSERT and UPDATE statements, a list of column names which should be present in the VALUES clause of the compiled statement. If None, all columns from the target table object are rendered.
- dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this ClauseElement's bound engine, if any.
- inline – Used for INSERT statements, for a dialect which does not support inline retrieval of newly generated primary key columns, will force the expression used to create the new primary key value to be rendered inline within the INSERT statement's VALUES clause. This typically refers to Sequence execution but may also refer to any server-side default generation function associated with a primary key Column.
- concat(other)¶
- inherited from the concat() method of ColumnOperators
Implement the 'concat' operator.
In a column context, produces the clause a || b, or uses the concat() operator on MySQL.
- contains(other, **kwargs)¶
- inherited from the contains() method of ColumnOperators
Implement the 'contains' operator.
In a column context, produces the clause LIKE '%<other>%'
- copy(**kw)¶
Create a copy of this Column, unitialized.
This is used in Table.tometadata.
- default = None¶
- desc()¶
- inherited from the desc() method of ColumnOperators
Produce a desc() clause against the parent object.
- description¶
- inherited from the description attribute of ColumnClause
- dispatch¶
alias of DDLEventsDispatch
- distinct()¶
- inherited from the distinct() method of ColumnOperators
Produce a distinct() clause against the parent object.
- endswith(other, **kwargs)¶
- inherited from the endswith() method of ColumnOperators
Implement the 'endswith' operator.
In a column context, produces the clause LIKE '%<other>'
- expression¶
- inherited from the expression attribute of ColumnElement
Return a column expression.
Part of the inspection interface; returns self.
- foreign_keys = []¶
- get_children(schema_visitor=False, **kwargs)¶
- ilike(other, escape=None)¶
- inherited from the ilike() method of ColumnOperators
Implement the ilike operator.
In a column context, produces the clause a ILIKE other.
Parameters: - other – expression to be compared
- escape –
optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.ilike("foo/%bar", escape="/")
See also
- in_(other)¶
- inherited from the in_() method of ColumnOperators
Implement the in operator.
In a column context, produces the clause a IN other. "other" may be a tuple/list of column expressions, or a select() construct.
- info¶
- inherited from the info attribute of SchemaItem
Info dictionary associated with the object, allowing user-defined data to be associated with this SchemaItem.
The dictionary is automatically generated when first accessed. It can also be specified in the constructor of some objects, such as Table and Column.
- is_(other)¶
- inherited from the is_() method of ColumnOperators
Implement the IS operator.
Normally, IS is generated automatically when comparing to a value of None, which resolves to NULL. However, explicit usage of IS may be desirable if comparing to boolean values on certain platforms.
New in version 0.7.9.
See also
- is_clause_element = True¶
- is_selectable = False¶
- isnot(other)¶
- inherited from the isnot() method of ColumnOperators
Implement the IS NOT operator.
Normally, IS NOT is generated automatically when comparing to a value of None, which resolves to NULL. However, explicit usage of IS NOT may be desirable if comparing to boolean values on certain platforms.
New in version 0.7.9.
See also
- label(name)¶
- inherited from the label() method of ColumnElement
Produce a column label, i.e. <columnname> AS <name>.
This is a shortcut to the label() function.
if 'name' is None, an anonymous label name will be generated.
- like(other, escape=None)¶
- inherited from the like() method of ColumnOperators
Implement the like operator.
In a column context, produces the clause a LIKE other.
Parameters: - other – expression to be compared
- escape –
optional escape character, renders the ESCAPE keyword, e.g.:
somecolumn.like("foo/%bar", escape="/")
See also
- match(other, **kwargs)¶
- inherited from the match() method of ColumnOperators
Implements the 'match' operator.
In a column context, this produces a MATCH clause, i.e. MATCH '<other>'. The allowed contents of other are database backend specific.
- notilike(other, escape=None)¶
- inherited from the notilike() method of ColumnOperators
implement the NOT ILIKE operator.
This is equivalent to using negation with ColumnOperators.ilike(), i.e. ~x.ilike(y).
New in version 0.8.
See also
- notin_(other)¶
- inherited from the notin_() method of ColumnOperators
implement the NOT IN operator.
This is equivalent to using negation with ColumnOperators.in_(), i.e. ~x.in_(y).
New in version 0.8.
See also
- notlike(other, escape=None)¶
- inherited from the notlike() method of ColumnOperators
implement the NOT LIKE operator.
This is equivalent to using negation with ColumnOperators.like(), i.e. ~x.like(y).
New in version 0.8.
See also
- nullsfirst()¶
- inherited from the nullsfirst() method of ColumnOperators
Produce a nullsfirst() clause against the parent object.
- nullslast()¶
- inherited from the nullslast() method of ColumnOperators
Produce a nullslast() clause against the parent object.
- onupdate = None¶
- op(opstring, precedence=0)¶
produce a generic operator function.
somecolumn * 5
This function can also be used to make bitwise operators explicit. For example:
is a bitwise AND of the value in somecolumn.
Parameters: - operator – a string which will be output as the infix operator between this element and the expression passed to the generated function.
- precedence –
precedence to apply to the operator, when parenthesizing expressions. A lower number will cause the expression to be parenthesized when applied against another operator with higher precedence. The default value of 0 is lower than all operators except for the comma (,) and AS operators. A value of 100 will be higher or equal to all operators, and -100 will be lower than or equal to all operators.
New in version 0.8: - added the 'precedence' argument.
- operate(op, *other, **kwargs)¶
- inherited from the operate() method of ColumnElement
- params(*optionaldict, **kwargs)¶
- inherited from the params() method of Immutable
- primary_key = False¶
- proxy_set¶
- inherited from the proxy_set attribute of ColumnElement
- quote = None¶
- references(column)¶
Return True if this Column references the given column via foreign key.
- reverse_operate(op, other, **kwargs)¶
- inherited from the reverse_operate() method of ColumnElement
- self_group(against=None)¶
- inherited from the self_group() method of ClauseElement
Apply a 'grouping' to this ClauseElement.
This method is overridden by subclasses to return a "grouping" construct, i.e. parenthesis. In particular it's used by "binary" expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select(). (Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code should never need to use this method directly. Note that SQLAlchemy's clause constructs take operator precedence into account - so parenthesis might not be needed, for example, in an expression like x OR (y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
- server_default = None¶
- server_onupdate = None¶
- shares_lineage(othercolumn)¶
- inherited from the shares_lineage() method of ColumnElement
Return True if the given ColumnElement has a common ancestor to this ColumnElement.
- startswith(other, **kwargs)¶
- inherited from the startswith() method of ColumnOperators
Implement the startwith operator.
In a column context, produces the clause LIKE '<other>%'
- supports_execution = False¶
- table¶
- inherited from the table attribute of ColumnClause
- timetuple = None¶
- type¶
- inherited from the type attribute of ColumnElement
- unique_params(*optionaldict, **kwargs)¶
- inherited from the unique_params() method of Immutable
- class sqlalchemy.schema.MetaData(bind=None, reflect=False, schema=None, quote_schema=None)¶
Bases: sqlalchemy.schema.SchemaItem
A collection of Table objects and their associated schema constructs.
Holds a collection of Table objects as well as an optional binding to an Engine or Connection. If bound, the Table objects in the collection and their columns may participate in implicit SQL execution.
The Table objects themselves are stored in the metadata.tables dictionary.
The bind property may be assigned to dynamically. A common pattern is to start unbound and then bind later when an engine is available:
metadata = MetaData() # define tables Table('mytable', metadata, ...) # connect to an engine later, perhaps after loading a URL from a # configuration file metadata.bind = an_engine
MetaData is a thread-safe object after tables have been explicitly defined or loaded via reflection.
See also:
Describing Databases with MetaData - Introduction to database metadata
- __init__(bind=None, reflect=False, schema=None, quote_schema=None)¶
Create a new MetaData object.
Parameters: - bind – An Engine or Connection to bind to. May also be a string or URL instance, these are passed to create_engine() and this MetaData will be bound to the resulting engine.
- reflect –
Optional, automatically load all tables from the bound database. Defaults to False. bind is required when this option is set.
Deprecated since version 0.8: Please use the MetaData.reflect() method.
- schema – The default schema to use for the Table, Sequence, and other objects associated with this MetaData. Defaults to None.
- quote_schema – Sets the quote_schema flag for those Table, Sequence, and other objects which make usage of the local schema name.
New in version 0.7.4: schema and quote_schema parameters.
- append_ddl_listener(event_name, listener)¶
Append a DDL event listener to this MetaData.
Deprecated. See DDLEvents.
- bind¶
An Engine or Connection to which this MetaData is bound.
Typically, a Engine is assigned to this attribute so that "implicit execution" may be used, or alternatively as a means of providing engine binding information to an ORM Session object:
engine = create_engine("someurl://") metadata.bind = engine
See also
Connectionless Execution, Implicit Execution - background on "bound metadata"
- clear()¶
Clear all Table objects from this MetaData.
- create_all(bind=None, tables=None, checkfirst=True)¶
Create all tables stored in this metadata.
Conditional by default, will not attempt to recreate tables already present in the target database.
Parameters: - bind – A Connectable used to access the database; if None, uses the existing bind on this MetaData, if any.
- tables – Optional list of Table objects, which is a subset of the total tables in the MetaData (others are ignored).
- checkfirst – Defaults to True, don't issue CREATEs for tables already present in the target database.
- drop_all(bind=None, tables=None, checkfirst=True)¶
Drop all tables stored in this metadata.
Conditional by default, will not attempt to drop tables not present in the target database.
Parameters: - bind – A Connectable used to access the database; if None, uses the existing bind on this MetaData, if any.
- tables – Optional list of Table objects, which is a subset of the total tables in the MetaData (others are ignored).
- checkfirst – Defaults to True, only issue DROPs for tables confirmed to be present in the target database.
- is_bound()¶
True if this MetaData is bound to an Engine or Connection.
- reflect(bind=None, schema=None, views=False, only=None)¶
Load all available table definitions from the database.
Automatically creates Table entries in this MetaData for any table available in the database but not yet present in the MetaData. May be called multiple times to pick up tables recently added to the database, however no special action is taken if a table in this MetaData no longer exists in the database.
Parameters: - bind – A Connectable used to access the database; if None, uses the existing bind on this MetaData, if any.
- schema – Optional, query and reflect tables from an alterate schema. If None, the schema associated with this MetaData is used, if any.
- views – If True, also reflect views.
- only –
Optional. Load only a sub-set of available named tables. May be specified as a sequence of names or a callable.
If a sequence of names is provided, only those tables will be reflected. An error is raised if a table is requested but not available. Named tables already present in this MetaData are ignored.
If a callable is provided, it will be used as a boolean predicate to filter the list of potential table names. The callable is called with a table name and this MetaData instance as positional arguments and should return a true value for any table to reflect.
- remove(table)¶
Remove the given Table object from this MetaData.
- sorted_tables¶
Returns a list of Table objects sorted in order of foreign key dependency.
The sorting will place Table objects that have dependencies first, before the dependencies themselves, representing the order in which they can be created. To get the order in which the tables would be dropped, use the reversed() Python built-in.
See also
- class sqlalchemy.schema.SchemaItem¶
Bases: sqlalchemy.events.SchemaEventTarget, sqlalchemy.sql.visitors.Visitable
Base class for items that define a database schema.
- get_children(**kwargs)¶
used to allow SchemaVisitor access
- info¶
Info dictionary associated with the object, allowing user-defined data to be associated with this SchemaItem.
The dictionary is automatically generated when first accessed. It can also be specified in the constructor of some objects, such as Table and Column.
- class sqlalchemy.schema.Table(*args, **kw)¶
Bases: sqlalchemy.schema.SchemaItem, sqlalchemy.sql.expression.TableClause
Represent a table in a database.
mytable = Table("mytable", metadata, Column('mytable_id', Integer, primary_key=True), Column('value', String(50)) )
The Table object constructs a unique instance of itself based on its name and optional schema name within the given MetaData object. Calling the Table constructor with the same name and same MetaData argument a second time will return the same Table object - in this way the Table constructor acts as a registry function.
See also:
Describing Databases with MetaData - Introduction to database metadata
Constructor arguments are as follows:
Parameters: - name –
The name of this table as represented in the database.
This property, along with the schema, indicates the singleton identity of this table in relation to its parent MetaData. Additional calls to Table with the same name, metadata, and schema name will return the same Table object.
Names which contain no upper case characters will be treated as case insensitive names, and will not be quoted unless they are a reserved word. Names with any number of upper case characters will be quoted and sent exactly. Note that this behavior applies even for databases which standardize upper case names as case insensitive such as Oracle.
- metadata – a MetaData object which will contain this table. The metadata is used as a point of association of this table with other tables which are referenced via foreign key. It also may be used to associate this table with a particular Connectable.
- *args – Additional positional arguments are used primarily to add the list of Column objects contained within this table. Similar to the style of a CREATE TABLE statement, other SchemaItem constructs may be added here, including PrimaryKeyConstraint, and ForeignKeyConstraint.
- autoload – Defaults to False: the Columns for this table should be reflected from the database. Usually there will be no Column objects in the constructor if this property is set.
- autoload_replace –
If True, when using autoload=True and extend_existing=True, replace Column objects already present in the Table that's in the MetaData registry with what's reflected. Otherwise, all existing columns will be excluded from the reflection process. Note that this does not impact Column objects specified in the same call to Table which includes autoload, those always take precedence. Defaults to True.
New in version 0.7.5.
- autoload_with – If autoload==True, this is an optional Engine or Connection instance to be used for the table reflection. If None, the underlying MetaData's bound connectable will be used.
- extend_existing –
When True, indicates that if this Table is already present in the given MetaData, apply further arguments within the constructor to the existing Table.
If extend_existing or keep_existing are not set, an error is raised if additional table modifiers are specified when the given Table is already present in the MetaData.
Changed in version 0.7.4: extend_existing will work in conjunction with autoload=True to run a new reflection operation against the database; new Column objects will be produced from database metadata to replace those existing with the same name, and additional Column objects not present in the Table will be added.
As is always the case with autoload=True, Column objects can be specified in the same Table constructor, which will take precedence. I.e.:
Table("mytable", metadata, Column('y', Integer), extend_existing=True, autoload=True, autoload_with=engine )
The above will overwrite all columns within mytable which are present in the database, except for y which will be used as is from the above definition. If the autoload_replace flag is set to False, no existing columns will be replaced.
- implicit_returning – True by default - indicates that RETURNING can be used by default to fetch newly inserted primary key values, for backends which support this. Note that create_engine() also provides an implicit_returning flag.
- include_columns – A list of strings indicating a subset of columns to be loaded via the autoload operation; table columns who aren't present in this list will not be represented on the resulting Table object. Defaults to None which indicates all columns should be reflected.
- info – Optional data dictionary which will be populated into the SchemaItem.info attribute of this object.
- keep_existing –
When True, indicates that if this Table is already present in the given MetaData, ignore further arguments within the constructor to the existing Table, and return the Table object as originally created. This is to allow a function that wishes to define a new Table on first call, but on subsequent calls will return the same Table, without any of the declarations (particularly constraints) being applied a second time. Also see extend_existing.
If extend_existing or keep_existing are not set, an error is raised if additional table modifiers are specified when the given Table is already present in the MetaData.
- listeners –
A list of tuples of the form (<eventname>, <fn>) which will be passed to event.listen() upon construction. This alternate hook to event.listen() allows the establishment of a listener function specific to this Table before the "autoload" process begins. Particularly useful for the DDLEvents.column_reflect() event:
def listen_for_reflect(table, column_info): "handle the column reflection event" # ... t = Table( 'sometable', autoload=True, listeners=[ ('column_reflect', listen_for_reflect) ])
- mustexist – When True, indicates that this Table must already be present in the given MetaData collection, else an exception is raised.
- prefixes – A list of strings to insert after CREATE in the CREATE TABLE statement. They will be separated by spaces.
- quote – Force quoting of this table's name on or off, corresponding to True or False. When left at its default of None, the column identifier will be quoted according to whether the name is case sensitive (identifiers with at least one upper case character are treated as case sensitive), or if it's a reserved word. This flag is only needed to force quoting of a reserved word which is not known by the SQLAlchemy dialect.
- quote_schema – same as 'quote' but applies to the schema identifier.
- schema – The schema name for this table, which is required if the table resides in a schema other than the default selected schema for the engine's database connection. Defaults to None.
- useexisting – Deprecated. Use extend_existing.
- __init__(*args, **kw)¶
Constructor for Table.
This method is a no-op. See the top-level documentation for Table for constructor arguments.
- add_is_dependent_on(table)¶
Add a 'dependency' for this Table.
This is another Table object which must be created first before this one can, or dropped after this one.
Usually, dependencies between tables are determined via ForeignKey objects. However, for other situations that create dependencies outside of foreign keys (rules, inheriting), this method can manually establish such a link.
- alias(name=None)¶
- inherited from the alias() method of FromClause
return an alias of this FromClause.
This is shorthand for calling:
from sqlalchemy import alias a = alias(self, name=name)
See alias() for details.
- append_column(column)¶
Append a Column to this Table.
The "key" of the newly added Column, i.e. the value of its .key attribute, will then be available in the .c collection of this Table, and the column definition will be included in any CREATE TABLE, SELECT, UPDATE, etc. statements generated from this Table construct.
Note that this does not change the definition of the table as it exists within any underlying database, assuming that table has already been created in the database. Relational databases support the addition of columns to existing tables using the SQL ALTER command, which would need to be emitted for an already-existing table that doesn't contain the newly added column.
- append_constraint(constraint)¶
Append a Constraint to this Table.
This has the effect of the constraint being included in any future CREATE TABLE statement, assuming specific DDL creation events have not been associated with the given Constraint object.
Note that this does not produce the constraint within the relational database automatically, for a table that already exists in the database. To add a constraint to an existing relational database table, the SQL ALTER command must be used. SQLAlchemy also provides the AddConstraint construct which can produce this SQL when invoked as an executable clause.
- append_ddl_listener(event_name, listener)¶
Append a DDL event listener to this Table.
Deprecated. See DDLEvents.
- bind¶
Return the connectable associated with this Table.
- c¶
- inherited from the c attribute of FromClause
An alias for the columns attribute.
- columns¶
- inherited from the columns attribute of FromClause
A named-based collection of ColumnElement objects maintained by this FromClause.
The columns, or c collection, is the gateway to the construction of SQL expressions using table-bound or other selectable-bound columns:
select([mytable]).where(mytable.c.somecolumn == 5)
- compare(other, **kw)¶
- inherited from the compare() method of ClauseElement
Compare this ClauseElement to the given ClauseElement.
Subclasses should override the default behavior, which is a straight identity comparison.
**kw are arguments consumed by subclass compare() methods and may be used to modify the criteria for comparison. (see ColumnElement)
- compile(bind=None, dialect=None, **kw)¶
- inherited from the compile() method of ClauseElement
Compile this SQL expression.
The return value is a Compiled object. Calling str() or unicode() on the returned value will yield a string representation of the result. The Compiled object also can return a dictionary of bind parameter names and values using the params accessor.
Parameters: - bind – An Engine or Connection from which a Compiled will be acquired. This argument takes precedence over this ClauseElement's bound engine, if any.
- column_keys – Used for INSERT and UPDATE statements, a list of column names which should be present in the VALUES clause of the compiled statement. If None, all columns from the target table object are rendered.
- dialect – A Dialect instance from which a Compiled will be acquired. This argument takes precedence over the bind argument as well as this ClauseElement's bound engine, if any.
- inline – Used for INSERT statements, for a dialect which does not support inline retrieval of newly generated primary key columns, will force the expression used to create the new primary key value to be rendered inline within the INSERT statement's VALUES clause. This typically refers to Sequence execution but may also refer to any server-side default generation function associated with a primary key Column.
- correspond_on_equivalents(column, equivalents)¶
- inherited from the correspond_on_equivalents() method of FromClause
Return corresponding_column for the given column, or if None search for a match in the given dictionary.
- corresponding_column(column, require_embedded=False)¶
- inherited from the corresponding_column() method of FromClause
Given a ColumnElement, return the exported ColumnElement object from this Selectable which corresponds to that original Column via a common ancestor column.
Parameters: - column – the target ColumnElement to be matched
- require_embedded – only return corresponding columns for
the given ColumnElement, if the given ColumnElement is actually present within a sub-element of this FromClause. Normally the column will match if it merely shares a common ancestor with one of the exported columns of this FromClause.
- count(whereclause=None, **params)¶
- inherited from the count() method of TableClause
return a SELECT COUNT generated against this TableClause.
- create(bind=None, checkfirst=False)¶
Issue a CREATE statement for this Table, using the given Connectable for connectivity.
See also MetaData.create_all().
- delete(whereclause=None, **kwargs)¶
- inherited from the delete() method of TableClause
Generate a delete() construct against this TableClause.
See delete() for argument and usage information.
- description¶
- inherited from the description attribute of TableClause
- dispatch¶
alias of DDLEventsDispatch
- drop(bind=None, checkfirst=False)¶
Issue a DROP statement for this Table, using the given Connectable for connectivity.
See also MetaData.drop_all().
- exists(bind=None)¶
Return True if this table exists.
- foreign_keys¶
- inherited from the foreign_keys attribute of FromClause
Return the collection of ForeignKey objects which this FromClause references.
- get_children(column_collections=True, schema_visitor=False, **kw)¶
- implicit_returning = False¶
- info¶
- inherited from the info attribute of SchemaItem
Info dictionary associated with the object, allowing user-defined data to be associated with this SchemaItem.
The dictionary is automatically generated when first accessed. It can also be specified in the constructor of some objects, such as Table and Column.
- insert(values=None, inline=False, **kwargs)¶
- inherited from the insert() method of TableClause
Generate an insert() construct against this TableClause.
See insert() for argument and usage information.
- is_clause_element = True¶
- is_derived_from(fromclause)¶
- inherited from the is_derived_from() method of FromClause
Return True if this FromClause is 'derived' from the given FromClause.
An example would be an Alias of a Table is derived from that Table.
- is_selectable = True¶
- join(right, onclause=None, isouter=False)¶
- inherited from the join() method of FromClause
return a join of this FromClause against another FromClause.
- key¶
- named_with_column = True¶
- outerjoin(right, onclause=None)¶
- inherited from the outerjoin() method of FromClause
return an outer join of this FromClause against another FromClause.
- params(*optionaldict, **kwargs)¶
- inherited from the params() method of Immutable
- primary_key¶
- inherited from the primary_key attribute of FromClause
Return the collection of Column objects which comprise the primary key of this FromClause.
- quote = None¶
- replace_selectable(old, alias)¶
- inherited from the replace_selectable() method of FromClause
replace all occurrences of FromClause 'old' with the given Alias object, returning a copy of this FromClause.
- schema = None¶
- select(whereclause=None, **params)¶
- inherited from the select() method of FromClause
return a SELECT of this FromClause.
See also
select() - general purpose method which allows for arbitrary column lists.
- selectable¶
- inherited from the selectable attribute of Selectable
- self_group(against=None)¶
- inherited from the self_group() method of ClauseElement
Apply a 'grouping' to this ClauseElement.
This method is overridden by subclasses to return a "grouping" construct, i.e. parenthesis. In particular it's used by "binary" expressions to provide a grouping around themselves when placed into a larger expression, as well as by select() constructs when placed into the FROM clause of another select(). (Note that subqueries should be normally created using the Select.alias() method, as many platforms require nested SELECT statements to be named).
As expressions are composed together, the application of self_group() is automatic - end-user code should never need to use this method directly. Note that SQLAlchemy's clause constructs take operator precedence into account - so parenthesis might not be needed, for example, in an expression like x OR (y AND z) - AND takes precedence over OR.
The base self_group() method of ClauseElement just returns self.
- supports_execution = False¶
- tometadata(metadata, schema=<symbol 'retain_schema>)¶
Return a copy of this Table associated with a different MetaData.
some_engine = create_engine("sqlite:///some.db") # create two metadata meta1 = MetaData() meta2 = MetaData() # load 'users' from the sqlite engine users_table = Table('users', meta1, autoload=True, autoload_with=some_engine) # create the same Table object for the plain metadata users_table_2 = users_table.tometadata(meta2)
- unique_params(*optionaldict, **kwargs)¶
- inherited from the unique_params() method of Immutable
- update(whereclause=None, values=None, inline=False, **kwargs)¶
- inherited from the update() method of TableClause
Generate an update() construct against this TableClause.
See update() for argument and usage information.
- name –
- class sqlalchemy.schema.ThreadLocalMetaData¶
Bases: sqlalchemy.schema.MetaData
A MetaData variant that presents a different bind in every thread.
Makes the bind property of the MetaData a thread-local value, allowing this collection of tables to be bound to different Engine implementations or connections in each thread.
The ThreadLocalMetaData starts off bound to None in each thread. Binds must be made explicitly by assigning to the bind property or using connect(). You can also re-bind dynamically multiple times per thread, just like a regular MetaData.
- __init__()¶
Construct a ThreadLocalMetaData.
- bind¶
The bound Engine or Connection for this thread.
This property may be assigned an Engine or Connection, or assigned a string or URL to automatically create a basic Engine for this bind with create_engine().
- dispose()¶
Dispose all bound engines, in all thread contexts.
- is_bound()¶
True if there is a bind for this thread.
Reflecting Database Objects¶
A Table object can be instructed to load information about itself from the corresponding database schema object already existing within the database. This process is called reflection. In the most simple case you need only specify the table name, a MetaData object, and the autoload=True flag. If the MetaData is not persistently bound, also add the autoload_with argument:
>>> messages = Table('messages', meta, autoload=True, autoload_with=engine)
>>> [c.name for c in messages.columns]
['message_id', 'message_name', 'date']
The above operation will use the given engine to query the database for information about the messages table, and will then generate Column, ForeignKey, and other objects corresponding to this information as though the Table object were hand-constructed in Python.
When tables are reflected, if a given table references another one via foreign key, a second Table object is created within the MetaData object representing the connection. Below, assume the table shopping_cart_items references a table named shopping_carts. Reflecting the shopping_cart_items table has the effect such that the shopping_carts table will also be loaded:
>>> shopping_cart_items = Table('shopping_cart_items', meta, autoload=True, autoload_with=engine)
>>> 'shopping_carts' in meta.tables:
The MetaData has an interesting "singleton-like" behavior such that if you requested both tables individually, MetaData will ensure that exactly one Table object is created for each distinct table name. The Table constructor actually returns to you the already-existing Table object if one already exists with the given name. Such as below, we can access the already generated shopping_carts table just by naming it:
shopping_carts = Table('shopping_carts', meta)
Of course, it's a good idea to use autoload=True with the above table regardless. This is so that the table's attributes will be loaded if they have not been already. The autoload operation only occurs for the table if it hasn't already been loaded; once loaded, new calls to Table with the same name will not re-issue any reflection queries.
Overriding Reflected Columns¶
Individual columns can be overridden with explicit values when reflecting tables; this is handy for specifying custom datatypes, constraints such as primary keys that may not be configured within the database, etc.:
>>> mytable = Table('mytable', meta,
... Column('id', Integer, primary_key=True), # override reflected 'id' to have primary key
... Column('mydata', Unicode(50)), # override reflected 'mydata' to be Unicode
... autoload=True)
Reflecting Views¶
The reflection system can also reflect views. Basic usage is the same as that of a table:
my_view = Table("some_view", metadata, autoload=True)
Above, my_view is a Table object with Column objects representing the names and types of each column within the view "some_view".
Usually, it's desired to have at least a primary key constraint when reflecting a view, if not foreign keys as well. View reflection doesn't extrapolate these constraints.
Use the "override" technique for this, specifying explicitly those columns which are part of the primary key or have foreign key constraints:
my_view = Table("some_view", metadata,
Column("view_id", Integer, primary_key=True),
Column("related_thing", Integer, ForeignKey("othertable.thing_id")),
Reflecting All Tables at Once¶
The MetaData object can also get a listing of tables and reflect the full set. This is achieved by using the reflect() method. After calling it, all located tables are present within the MetaData object's dictionary of tables:
meta = MetaData()
users_table = meta.tables['users']
addresses_table = meta.tables['addresses']
metadata.reflect() also provides a handy way to clear or delete all the rows in a database:
meta = MetaData()
for table in reversed(meta.sorted_tables):
Fine Grained Reflection with Inspector¶
A low level interface which provides a backend-agnostic system of loading lists of schema, table, column, and constraint descriptions from a given database is also available. This is known as the "Inspector":
from sqlalchemy import create_engine
from sqlalchemy.engine import reflection
engine = create_engine('...')
insp = reflection.Inspector.from_engine(engine)
print insp.get_table_names()
- class sqlalchemy.engine.reflection.Inspector(bind)¶
Bases: object
Performs database schema inspection.
The Inspector acts as a proxy to the reflection methods of the Dialect, providing a consistent interface as well as caching support for previously fetched metadata.
A Inspector object is usually created via the inspect() function:
from sqlalchemy import inspect, create_engine engine = create_engine('...') insp = inspect(engine)
The inspection method above is equivalent to using the Inspector.from_engine() method, i.e.:
engine = create_engine('...') insp = Inspector.from_engine(engine)
Where above, the Dialect may opt to return an Inspector subclass that provides additional methods specific to the dialect's target database.
- __init__(bind)¶
Initialize a new Inspector.
Parameters: bind – a Connectable, which is typically an instance of Engine or Connection. For a dialect-specific instance of Inspector, see Inspector.from_engine()
- default_schema_name¶
Return the default schema name presented by the dialect for the current engine's database user.
E.g. this is typically public for Postgresql and dbo for SQL Server.
- classmethod from_engine(bind)¶
Construct a new dialect-specific Inspector object from the given engine or connection.
Parameters: bind – a Connectable, which is typically an instance of Engine or Connection. This method differs from direct a direct constructor call of Inspector in that the Dialect is given a chance to provide a dialect-specific Inspector instance, which may provide additional methods.
See the example at Inspector.
- get_columns(table_name, schema=None, **kw)¶
Return information about columns in table_name.
Given a string table_name and an optional string schema, return column information as a list of dicts with these keys:
- name
- the column's name
- type
- TypeEngine
- nullable
- boolean
- default
- the column's default value
- attrs
- dict containing optional column attributes
- get_foreign_keys(table_name, schema=None, **kw)¶
Return information about foreign_keys in table_name.
Given a string table_name, and an optional string schema, return foreign key information as a list of dicts with these keys:
- constrained_columns
- a list of column names that make up the foreign key
- referred_schema
- the name of the referred schema
- referred_table
- the name of the referred table
- referred_columns
- a list of column names in the referred table that correspond to constrained_columns
- name
- optional name of the foreign key constraint.
- **kw
- other options passed to the dialect's get_foreign_keys() method.
- get_indexes(table_name, schema=None, **kw)¶
Return information about indexes in table_name.
Given a string table_name and an optional string schema, return index information as a list of dicts with these keys:
- name
- the index's name
- column_names
- list of column names in order
- unique
- boolean
- **kw
- other options passed to the dialect's get_indexes() method.
- get_pk_constraint(table_name, schema=None, **kw)¶
Return information about primary key constraint on table_name.
Given a string table_name, and an optional string schema, return primary key information as a dictionary with these keys:
- constrained_columns
- a list of column names that make up the primary key
- name
- optional name of the primary key constraint.
- get_primary_keys(table_name, schema=None, **kw)¶
Return information about primary keys in table_name.
Deprecated since version 0.7: Call to deprecated method get_primary_keys. Use get_pk_constraint instead.
Given a string table_name, and an optional string schema, return primary key information as a list of column names.
- get_schema_names()¶
Return all schema names.
- get_table_names(schema=None, order_by=None)¶
Return all table names in referred to within a particular schema.
The names are expected to be real tables only, not views. Views are instead returned using the get_view_names() method.
Parameters: - schema – Schema name. If schema is left at None, the database's default schema is used, else the named schema is searched. If the database does not support named schemas, behavior is undefined if schema is not passed as None.
- order_by –
Optional, may be the string "foreign_key" to sort the result on foreign key dependencies.
Changed in version 0.8: the "foreign_key" sorting sorts tables in order of dependee to dependent; that is, in creation order, rather than in drop order. This is to maintain consistency with similar features such as MetaData.sorted_tables and util.sort_tables().
See also
- get_table_options(table_name, schema=None, **kw)¶
Return a dictionary of options specified when the table of the given name was created.
This currently includes some options that apply to MySQL tables.
- get_view_definition(view_name, schema=None)¶
Return definition for view_name.
Parameters: schema – Optional, retrieve names from a non-default schema.
- get_view_names(schema=None)¶
Return all view names in schema.
Parameters: schema – Optional, retrieve names from a non-default schema.
- reflecttable(table, include_columns, exclude_columns=())¶
Given a Table object, load its internal constructs based on introspection.
This is the underlying method used by most dialects to produce table reflection. Direct usage is like:
from sqlalchemy import create_engine, MetaData, Table from sqlalchemy.engine import reflection engine = create_engine('...') meta = MetaData() user_table = Table('user', meta) insp = Inspector.from_engine(engine) insp.reflecttable(user_table, None)
Parameters: - table – a Table instance.
- include_columns – a list of string column names to include in the reflection process. If None, all columns are reflected.
Column Insert/Update Defaults¶
SQLAlchemy provides a very rich featureset regarding column level events which take place during INSERT and UPDATE statements. Options include:
- Scalar values used as defaults during INSERT and UPDATE operations
- Python functions which execute upon INSERT and UPDATE operations
- SQL expressions which are embedded in INSERT statements (or in some cases execute beforehand)
- SQL expressions which are embedded in UPDATE statements
- Server side default values used during INSERT
- Markers for server-side triggers used during UPDATE
The general rule for all insert/update defaults is that they only take effect if no value for a particular column is passed as an execute() parameter; otherwise, the given value is used.
Scalar Defaults¶
The simplest kind of default is a scalar value used as the default value of a column:
Table("mytable", meta,
Column("somecolumn", Integer, default=12)
Above, the value "12" will be bound as the column value during an INSERT if no other value is supplied.
A scalar value may also be associated with an UPDATE statement, though this is not very common (as UPDATE statements are usually looking for dynamic defaults):
Table("mytable", meta,
Column("somecolumn", Integer, onupdate=25)
Python-Executed Functions¶
The default and onupdate keyword arguments also accept Python functions. These functions are invoked at the time of insert or update if no other value for that column is supplied, and the value returned is used for the column's value. Below illustrates a crude "sequence" that assigns an incrementing counter to a primary key column:
# a function which counts upwards
i = 0
def mydefault():
global i
i += 1
return i
t = Table("mytable", meta,
Column('id', Integer, primary_key=True, default=mydefault),
It should be noted that for real "incrementing sequence" behavior, the built-in capabilities of the database should normally be used, which may include sequence objects or other autoincrementing capabilities. For primary key columns, SQLAlchemy will in most cases use these capabilities automatically. See the API documentation for Column including the autoincrement flag, as well as the section on Sequence later in this chapter for background on standard primary key generation techniques.
To illustrate onupdate, we assign the Python datetime function now to the onupdate attribute:
import datetime
t = Table("mytable", meta,
Column('id', Integer, primary_key=True),
# define 'last_updated' to be populated with datetime.now()
Column('last_updated', DateTime, onupdate=datetime.datetime.now),
When an update statement executes and no value is passed for last_updated, the datetime.datetime.now() Python function is executed and its return value used as the value for last_updated. Notice that we provide now as the function itself without calling it (i.e. there are no parenthesis following) - SQLAlchemy will execute the function at the time the statement executes.
Context-Sensitive Default Functions¶
The Python functions used by default and onupdate may also make use of the current statement's context in order to determine a value. The context of a statement is an internal SQLAlchemy object which contains all information about the statement being executed, including its source expression, the parameters associated with it and the cursor. The typical use case for this context with regards to default generation is to have access to the other values being inserted or updated on the row. To access the context, provide a function that accepts a single context argument:
def mydefault(context):
return context.current_parameters['counter'] + 12
t = Table('mytable', meta,
Column('counter', Integer),
Column('counter_plus_twelve', Integer, default=mydefault, onupdate=mydefault)
Above we illustrate a default function which will execute for all INSERT and UPDATE statements where a value for counter_plus_twelve was otherwise not provided, and the value will be that of whatever value is present in the execution for the counter column, plus the number 12.
While the context object passed to the default function has many attributes, the current_parameters member is a special member provided only during the execution of a default function for the purposes of deriving defaults from its existing values. For a single statement that is executing many sets of bind parameters, the user-defined function is called for each set of parameters, and current_parameters will be provided with each individual parameter set for each execution.
SQL Expressions¶
The "default" and "onupdate" keywords may also be passed SQL expressions, including select statements or direct function calls:
t = Table("mytable", meta,
Column('id', Integer, primary_key=True),
# define 'create_date' to default to now()
Column('create_date', DateTime, default=func.now()),
# define 'key' to pull its default from the 'keyvalues' table
Column('key', String(20), default=keyvalues.select(keyvalues.c.type='type1', limit=1)),
# define 'last_modified' to use the current_timestamp SQL function on update
Column('last_modified', DateTime, onupdate=func.utc_timestamp())
Above, the create_date column will be populated with the result of the now() SQL function (which, depending on backend, compiles into NOW() or CURRENT_TIMESTAMP in most cases) during an INSERT statement, and the key column with the result of a SELECT subquery from another table. The last_modified column will be populated with the value of UTC_TIMESTAMP(), a function specific to MySQL, when an UPDATE statement is emitted for this table.
Note that when using func functions, unlike when using Python datetime functions we do call the function, i.e. with parenthesis "()" - this is because what we want in this case is the return value of the function, which is the SQL expression construct that will be rendered into the INSERT or UPDATE statement.
The above SQL functions are usually executed "inline" with the INSERT or UPDATE statement being executed, meaning, a single statement is executed which embeds the given expressions or subqueries within the VALUES or SET clause of the statement. Although in some cases, the function is "pre-executed" in a SELECT statement of its own beforehand. This happens when all of the following is true:
- the column is a primary key column
- the database dialect does not support a usable cursor.lastrowid accessor (or equivalent); this currently includes PostgreSQL, Oracle, and Firebird, as well as some MySQL dialects.
- the dialect does not support the "RETURNING" clause or similar, or the implicit_returning flag is set to False for the dialect. Dialects which support RETURNING currently include Postgresql, Oracle, Firebird, and MS-SQL.
- the statement is a single execution, i.e. only supplies one set of parameters and doesn't use "executemany" behavior
- the inline=True flag is not set on the Insert() or Update() construct, and the statement has not defined an explicit returning() clause.
Whether or not the default generation clause "pre-executes" is not something that normally needs to be considered, unless it is being addressed for performance reasons.
When the statement is executed with a single set of parameters (that is, it is not an "executemany" style execution), the returned ResultProxy will contain a collection accessible via result.postfetch_cols() which contains a list of all Column objects which had an inline-executed default. Similarly, all parameters which were bound to the statement, including all Python and SQL expressions which were pre-executed, are present in the last_inserted_params() or last_updated_params() collections on ResultProxy. The inserted_primary_key collection contains a list of primary key values for the row inserted (a list so that single-column and composite-column primary keys are represented in the same format).
Server Side Defaults¶
A variant on the SQL expression default is the server_default, which gets placed in the CREATE TABLE statement during a create() operation:
t = Table('test', meta,
Column('abc', String(20), server_default='abc'),
Column('created_at', DateTime, server_default=text("sysdate"))
A create call for the above table will produce:
CREATE TABLE test (
abc varchar(20) default 'abc',
created_at datetime default sysdate
The behavior of server_default is similar to that of a regular SQL default; if it's placed on a primary key column for a database which doesn't have a way to "postfetch" the ID, and the statement is not "inlined", the SQL expression is pre-executed; otherwise, SQLAlchemy lets the default fire off on the database side normally.
Triggered Columns¶
Columns with values set by a database trigger or other external process may be called out using FetchedValue as a marker:
t = Table('test', meta,
Column('abc', String(20), server_default=FetchedValue()),
Column('def', String(20), server_onupdate=FetchedValue())
Changed in version 0.8.0b2,0.7.10: The for_update argument on FetchedValue is set automatically when specified as the server_onupdate argument. If using an older version, specify the onupdate above as server_onupdate=FetchedValue(for_update=True).
These markers do not emit a "default" clause when the table is created, however they do set the same internal flags as a static server_default clause, providing hints to higher-level tools that a "post-fetch" of these rows should be performed after an insert or update.
It's generally not appropriate to use FetchedValue in conjunction with a primary key column, particularly when using the ORM or any other scenario where the ResultProxy.inserted_primary_key attribute is required. This is becaue the "post-fetch" operation requires that the primary key value already be available, so that the row can be selected on its primary key.
For a server-generated primary key value, all databases provide special accessors or other techniques in order to acquire the "last inserted primary key" column of a table. These mechanisms aren't affected by the presence of FetchedValue. For special situations where triggers are used to generate primary key values, and the database in use does not support the RETURNING clause, it may be necessary to forego the usage of the trigger and instead apply the SQL expression or function as a "pre execute" expression:
t = Table('test', meta,
Column('abc', MyType, default=func.generate_new_value(), primary_key=True)
Where above, when Table.insert() is used, the func.generate_new_value() expression will be pre-executed in the context of a scalar SELECT statement, and the new value will be applied to the subsequent INSERT, while at the same time being made available to the ResultProxy.inserted_primary_key attribute.
Defining Sequences¶
SQLAlchemy represents database sequences using the Sequence object, which is considered to be a special case of "column default". It only has an effect on databases which have explicit support for sequences, which currently includes Postgresql, Oracle, and Firebird. The Sequence object is otherwise ignored.
The Sequence may be placed on any column as a "default" generator to be used during INSERT operations, and can also be configured to fire off during UPDATE operations if desired. It is most commonly used in conjunction with a single integer primary key column:
table = Table("cartitems", meta,
Column("cart_id", Integer, Sequence('cart_id_seq'), primary_key=True),
Column("description", String(40)),
Column("createdate", DateTime())
Where above, the table "cartitems" is associated with a sequence named "cart_id_seq". When INSERT statements take place for "cartitems", and no value is passed for the "cart_id" column, the "cart_id_seq" sequence will be used to generate a value.
When the Sequence is associated with a table, CREATE and DROP statements issued for that table will also issue CREATE/DROP for the sequence object as well, thus "bundling" the sequence object with its parent table.
The Sequence object also implements special functionality to accommodate Postgresql's SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a Table object defines a Sequence on its primary key column so that it works with Oracle and Firebird, the Sequence would get in the way of the "implicit" sequence that PG would normally use. For this use case, add the flag optional=True to the Sequence object - this indicates that the Sequence should only be used if the database provides no other option for generating primary key identifiers.
The Sequence object also has the ability to be executed standalone like a SQL expression, which has the effect of calling its "next value" function:
seq = Sequence('some_sequence')
nextid = connection.execute(seq)
Default Objects API¶
- class sqlalchemy.schema.ColumnDefault(arg, **kwargs)¶
Bases: sqlalchemy.schema.DefaultGenerator
A plain default value on a column.
This could correspond to a constant, a callable function, or a SQL clause.
ColumnDefault is generated automatically whenever the default, onupdate arguments of Column are used. A ColumnDefault can be passed positionally as well.
For example, the following:
Column('foo', Integer, default=50)
Is equivalent to:
Column('foo', Integer, ColumnDefault(50))
- class sqlalchemy.schema.DefaultClause(arg, for_update=False, _reflected=False)¶
Bases: sqlalchemy.schema.FetchedValue
A DDL-specified DEFAULT column value.
DefaultClause is a FetchedValue that also generates a "DEFAULT" clause when "CREATE TABLE" is emitted.
DefaultClause is generated automatically whenever the server_default, server_onupdate arguments of Column are used. A DefaultClause can be passed positionally as well.
For example, the following:
Column('foo', Integer, server_default="50")
Is equivalent to:
Column('foo', Integer, DefaultClause("50"))
- class sqlalchemy.schema.DefaultGenerator(for_update=False)¶
Bases: sqlalchemy.schema._NotAColumnExpr, sqlalchemy.schema.SchemaItem
Base class for column default values.
- class sqlalchemy.schema.FetchedValue(for_update=False)¶
Bases: sqlalchemy.schema._NotAColumnExpr, sqlalchemy.events.SchemaEventTarget
A marker for a transparent database-side default.
Use FetchedValue when the database is configured to provide some automatic default for a column.
Column('foo', Integer, FetchedValue())
Would indicate that some trigger or default generator will create a new value for the foo column during an INSERT.
See also
- class sqlalchemy.schema.PassiveDefault(*arg, **kw)¶
Bases: sqlalchemy.schema.DefaultClause
A DDL-specified DEFAULT column value.
Deprecated since version 0.6: PassiveDefault is deprecated. Use DefaultClause.
- class sqlalchemy.schema.Sequence(name, start=None, increment=None, schema=None, optional=False, quote=None, metadata=None, quote_schema=None, for_update=False)¶
Bases: sqlalchemy.schema.DefaultGenerator
Represents a named database sequence.
The Sequence object represents the name and configurational parameters of a database sequence. It also represents a construct that can be "executed" by a SQLAlchemy Engine or Connection, rendering the appropriate "next value" function for the target database and returning a result.
The Sequence is typically associated with a primary key column:
some_table = Table('some_table', metadata, Column('id', Integer, Sequence('some_table_seq'), primary_key=True) )
When CREATE TABLE is emitted for the above Table, if the target platform supports sequences, a CREATE SEQUENCE statement will be emitted as well. For platforms that don't support sequences, the Sequence construct is ignored.
See also: CreateSequence DropSequence
- __init__(name, start=None, increment=None, schema=None, optional=False, quote=None, metadata=None, quote_schema=None, for_update=False)¶
Construct a Sequence object.
Parameters: - name – The name of the sequence.
- start – the starting index of the sequence. This value is used when the CREATE SEQUENCE command is emitted to the database as the value of the "START WITH" clause. If None, the clause is omitted, which on most platforms indicates a starting value of 1.
- increment – the increment value of the sequence. This value is used when the CREATE SEQUENCE command is emitted to the database as the value of the "INCREMENT BY" clause. If None, the clause is omitted, which on most platforms indicates an increment of 1.
- schema – Optional schema name for the sequence, if located in a schema other than the default.
- optional – boolean value, when True, indicates that this Sequence object only needs to be explicitly generated on backends that don't provide another way to generate primary key identifiers. Currently, it essentially means, "don't create this sequence on the Postgresql backend, where the SERIAL keyword creates a sequence for us automatically".
- quote – boolean value, when True or False, explicitly forces quoting of the schema name on or off. When left at its default of None, normal quoting rules based on casing and reserved words take place.
- metadata –
optional MetaData object which will be associated with this Sequence. A Sequence that is associated with a MetaData gains access to the bind of that MetaData, meaning the Sequence.create() and Sequence.drop() methods will make usage of that engine automatically.
Changed in version 0.7: Additionally, the appropriate CREATE SEQUENCE/ DROP SEQUENCE DDL commands will be emitted corresponding to this Sequence when MetaData.create_all() and MetaData.drop_all() are invoked.
Note that when a Sequence is applied to a Column, the Sequence is automatically associated with the MetaData object of that column's parent Table, when that association is made. The Sequence will then be subject to automatic CREATE SEQUENCE/DROP SEQUENCE corresponding to when the Table object itself is created or dropped, rather than that of the MetaData object overall.
- for_update – Indicates this Sequence, when associated with a Column, should be invoked for UPDATE statements on that column's table, rather than for INSERT statements, when no value is otherwise present for that column in the statement.
- create(bind=None, checkfirst=True)¶
Creates this sequence in the database.
- drop(bind=None, checkfirst=True)¶
Drops this sequence from the database.
- next_value()¶
Return a next_value function element which will render the appropriate increment function for this Sequence within any SQL expression.
Defining Constraints and Indexes¶
Defining Foreign Keys¶
A foreign key in SQL is a table-level construct that constrains one or more columns in that table to only allow values that are present in a different set of columns, typically but not always located on a different table. We call the columns which are constrained the foreign key columns and the columns which they are constrained towards the referenced columns. The referenced columns almost always define the primary key for their owning table, though there are exceptions to this. The foreign key is the "joint" that connects together pairs of rows which have a relationship with each other, and SQLAlchemy assigns very deep importance to this concept in virtually every area of its operation.
In SQLAlchemy as well as in DDL, foreign key constraints can be defined as additional attributes within the table clause, or for single-column foreign keys they may optionally be specified within the definition of a single column. The single column foreign key is more common, and at the column level is specified by constructing a ForeignKey object as an argument to a Column object:
user_preference = Table('user_preference', metadata,
Column('pref_id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey("user.user_id"), nullable=False),
Column('pref_name', String(40), nullable=False),
Column('pref_value', String(100))
Above, we define a new table user_preference for which each row must contain a value in the user_id column that also exists in the user table's user_id column.
The argument to ForeignKey is most commonly a string of the form <tablename>.<columnname>, or for a table in a remote schema or "owner" of the form <schemaname>.<tablename>.<columnname>. It may also be an actual Column object, which as we'll see later is accessed from an existing Table object via its c collection:
The advantage to using a string is that the in-python linkage between user and user_preference is resolved only when first needed, so that table objects can be easily spread across multiple modules and defined in any order.
Foreign keys may also be defined at the table level, using the ForeignKeyConstraint object. This object can describe a single- or multi-column foreign key. A multi-column foreign key is known as a composite foreign key, and almost always references a table that has a composite primary key. Below we define a table invoice which has a composite primary key:
invoice = Table('invoice', metadata,
Column('invoice_id', Integer, primary_key=True),
Column('ref_num', Integer, primary_key=True),
Column('description', String(60), nullable=False)
And then a table invoice_item with a composite foreign key referencing invoice:
invoice_item = Table('invoice_item', metadata,
Column('item_id', Integer, primary_key=True),
Column('item_name', String(60), nullable=False),
Column('invoice_id', Integer, nullable=False),
Column('ref_num', Integer, nullable=False),
ForeignKeyConstraint(['invoice_id', 'ref_num'], ['invoice.invoice_id', 'invoice.ref_num'])
It's important to note that the ForeignKeyConstraint is the only way to define a composite foreign key. While we could also have placed individual ForeignKey objects on both the invoice_item.invoice_id and invoice_item.ref_num columns, SQLAlchemy would not be aware that these two values should be paired together - it would be two individual foreign key constraints instead of a single composite foreign key referencing two columns.
Creating/Dropping Foreign Key Constraints via ALTER¶
In all the above examples, the ForeignKey object causes the "REFERENCES" keyword to be added inline to a column definition within a "CREATE TABLE" statement when create_all() is issued, and ForeignKeyConstraint invokes the "CONSTRAINT" keyword inline with "CREATE TABLE". There are some cases where this is undesireable, particularly when two tables reference each other mutually, each with a foreign key referencing the other. In such a situation at least one of the foreign key constraints must be generated after both tables have been built. To support such a scheme, ForeignKey and ForeignKeyConstraint offer the flag use_alter=True. When using this flag, the constraint will be generated using a definition similar to "ALTER TABLE <tablename> ADD CONSTRAINT <name> ...". Since a name is required, the name attribute must also be specified. For example:
node = Table('node', meta,
Column('node_id', Integer, primary_key=True),
Column('primary_element', Integer,
ForeignKey('element.element_id', use_alter=True, name='fk_node_element_id')
element = Table('element', meta,
Column('element_id', Integer, primary_key=True),
Column('parent_node_id', Integer),
ON UPDATE and ON DELETE¶
Most databases support cascading of foreign key values, that is the when a parent row is updated the new value is placed in child rows, or when the parent row is deleted all corresponding child rows are set to null or deleted. In data definition language these are specified using phrases like "ON UPDATE CASCADE", "ON DELETE CASCADE", and "ON DELETE SET NULL", corresponding to foreign key constraints. The phrase after "ON UPDATE" or "ON DELETE" may also other allow other phrases that are specific to the database in use. The ForeignKey and ForeignKeyConstraint objects support the generation of this clause via the onupdate and ondelete keyword arguments. The value is any string which will be output after the appropriate "ON UPDATE" or "ON DELETE" phrase:
child = Table('child', meta,
Column('id', Integer,
ForeignKey('parent.id', onupdate="CASCADE", ondelete="CASCADE"),
composite = Table('composite', meta,
Column('id', Integer, primary_key=True),
Column('rev_id', Integer),
Column('note_id', Integer),
['rev_id', 'note_id'],
['revisions.id', 'revisions.note_id'],
onupdate="CASCADE", ondelete="SET NULL"
Note that these clauses are not supported on SQLite, and require InnoDB tables when used with MySQL. They may also not be supported on other databases.
UNIQUE Constraint¶
Unique constraints can be created anonymously on a single column using the unique keyword on Column. Explicitly named unique constraints and/or those with multiple columns are created via the UniqueConstraint table-level construct.
meta = MetaData()
mytable = Table('mytable', meta,
# per-column anonymous unique constraint
Column('col1', Integer, unique=True),
Column('col2', Integer),
Column('col3', Integer),
# explicit/composite unique constraint. 'name' is optional.
UniqueConstraint('col2', 'col3', name='uix_1')
CHECK Constraint¶
Check constraints can be named or unnamed and can be created at the Column or Table level, using the CheckConstraint construct. The text of the check constraint is passed directly through to the database, so there is limited "database independent" behavior. Column level check constraints generally should only refer to the column to which they are placed, while table level constraints can refer to any columns in the table.
Note that some databases do not actively support check constraints such as MySQL.
meta = MetaData()
mytable = Table('mytable', meta,
# per-column CHECK constraint
Column('col1', Integer, CheckConstraint('col1>5')),
Column('col2', Integer),
Column('col3', Integer),
# table level CHECK constraint. 'name' is optional.
CheckConstraint('col2 > col3 + 5', name='check1')
CREATE TABLE mytable (
col1 INTEGER CHECK (col1>5),
col2 INTEGER,
col3 INTEGER,
CONSTRAINT check1 CHECK (col2 > col3 + 5)
Setting up Constraints when using the Declarative ORM Extension¶
The Table is the SQLAlchemy Core construct that allows one to define table metadata, which among other things can be used by the SQLAlchemy ORM as a target to map a class. The Declarative extension allows the Table object to be created automatically, given the contents of the table primarily as a mapping of Column objects.
To apply table-level constraint objects such as ForeignKeyConstraint to a table defined using Declarative, use the __table_args__ attribute, described at Table Configuration.
Constraints API¶
- class sqlalchemy.schema.Constraint(name=None, deferrable=None, initially=None, _create_rule=None, **kw)¶
Bases: sqlalchemy.schema.SchemaItem
A table-level SQL constraint.
- class sqlalchemy.schema.CheckConstraint(sqltext, name=None, deferrable=None, initially=None, table=None, _create_rule=None, _autoattach=True)¶
Bases: sqlalchemy.schema.Constraint
A table- or column-level CHECK constraint.
Can be included in the definition of a Table or Column.
- class sqlalchemy.schema.ColumnCollectionConstraint(*columns, **kw)¶
Bases: sqlalchemy.schema.ColumnCollectionMixin, sqlalchemy.schema.Constraint
A constraint that proxies a ColumnCollection.
- class sqlalchemy.schema.ForeignKey(column, _constraint=None, use_alter=False, name=None, onupdate=None, ondelete=None, deferrable=None, schema=None, initially=None, link_to_name=False, match=None)¶
Bases: sqlalchemy.schema.SchemaItem
Defines a dependency between two columns.
ForeignKey is specified as an argument to a Column object, e.g.:
t = Table("remote_table", metadata, Column("remote_id", ForeignKey("main_table.id")) )
Note that ForeignKey is only a marker object that defines a dependency between two columns. The actual constraint is in all cases represented by the ForeignKeyConstraint object. This object will be generated automatically when a ForeignKey is associated with a Column which in turn is associated with a Table. Conversely, when ForeignKeyConstraint is applied to a Table, ForeignKey markers are automatically generated to be present on each associated Column, which are also associated with the constraint object.
Note that you cannot define a "composite" foreign key constraint, that is a constraint between a grouping of multiple parent/child columns, using ForeignKey objects. To define this grouping, the ForeignKeyConstraint object must be used, and applied to the Table. The associated ForeignKey objects are created automatically.
The ForeignKey objects associated with an individual Column object are available in the foreign_keys collection of that column.
Further examples of foreign key configuration are in Defining Foreign Keys.
- __init__(column, _constraint=None, use_alter=False, name=None, onupdate=None, ondelete=None, deferrable=None, schema=None, initially=None, link_to_name=False, match=None)¶
Construct a column-level FOREIGN KEY.
The ForeignKey object when constructed generates a ForeignKeyConstraint which is associated with the parent Table object's collection of constraints.
Parameters: - column –
A single target column for the key relationship. A Column object or a column name as a string: tablename.columnkey or schema.tablename.columnkey. columnkey is the key which has been assigned to the column (defaults to the column name itself), unless link_to_name is True in which case the rendered name of the column is used.
New in version 0.7.4: Note that if the schema name is not included, and the underlying MetaData has a "schema", that value will be used.
- name – Optional string. An in-database name for the key if constraint is not provided.
- onupdate – Optional string. If set, emit ON UPDATE <value> when issuing DDL for this constraint. Typical values include CASCADE, DELETE and RESTRICT.
- ondelete – Optional string. If set, emit ON DELETE <value> when issuing DDL for this constraint. Typical values include CASCADE, DELETE and RESTRICT.
- deferrable – Optional bool. If set, emit DEFERRABLE or NOT DEFERRABLE when issuing DDL for this constraint.
- initially – Optional string. If set, emit INITIALLY <value> when issuing DDL for this constraint.
- link_to_name – if True, the string name given in column is the rendered name of the referenced column, not its locally assigned key.
- use_alter – passed to the underlying ForeignKeyConstraint to indicate the constraint should be generated/dropped externally from the CREATE TABLE/ DROP TABLE statement. See that classes' constructor for details.
- match – Optional string. If set, emit MATCH <value> when issuing DDL for this constraint. Typical values include SIMPLE, PARTIAL and FULL.
- column –
- column¶
Return the target Column referenced by this ForeignKey.
If this ForeignKey was created using a string-based target column specification, this attribute will on first access initiate a resolution process to locate the referenced remote Column. The resolution process traverses to the parent Column, Table, and MetaData to proceed - if any of these aren't yet present, an error is raised.
- copy(schema=None)¶
Produce a copy of this ForeignKey object.
The new ForeignKey will not be bound to any Column.
This method is usually used by the internal copy procedures of Column, Table, and MetaData.
Parameters: schema – The returned ForeignKey will reference the original table and column name, qualified by the given string schema name.
- get_referent(table)¶
Return the Column in the given Table referenced by this ForeignKey.
Returns None if this ForeignKey does not reference the given Table.
- references(table)¶
Return True if the given Table is referenced by this ForeignKey.
- target_fullname¶
Return a string based 'column specification' for this ForeignKey.
This is usually the equivalent of the string-based "tablename.colname" argument first passed to the object's constructor.
- class sqlalchemy.schema.ForeignKeyConstraint(columns, refcolumns, name=None, onupdate=None, ondelete=None, deferrable=None, initially=None, use_alter=False, link_to_name=False, match=None, table=None)¶
Bases: sqlalchemy.schema.Constraint
A table-level FOREIGN KEY constraint.
Defines a single column or composite FOREIGN KEY ... REFERENCES constraint. For a no-frills, single column foreign key, adding a ForeignKey to the definition of a Column is a shorthand equivalent for an unnamed, single column ForeignKeyConstraint.
Examples of foreign key configuration are in Defining Foreign Keys.
- __init__(columns, refcolumns, name=None, onupdate=None, ondelete=None, deferrable=None, initially=None, use_alter=False, link_to_name=False, match=None, table=None)¶
Construct a composite-capable FOREIGN KEY.
Parameters: - columns – A sequence of local column names. The named columns must be defined and present in the parent Table. The names should match the key given to each column (defaults to the name) unless link_to_name is True.
- refcolumns – A sequence of foreign column names or Column objects. The columns must all be located within the same Table.
- name – Optional, the in-database name of the key.
- onupdate – Optional string. If set, emit ON UPDATE <value> when issuing DDL for this constraint. Typical values include CASCADE, DELETE and RESTRICT.
- ondelete – Optional string. If set, emit ON DELETE <value> when issuing DDL for this constraint. Typical values include CASCADE, DELETE and RESTRICT.
- deferrable – Optional bool. If set, emit DEFERRABLE or NOT DEFERRABLE when issuing DDL for this constraint.
- initially – Optional string. If set, emit INITIALLY <value> when issuing DDL for this constraint.
- link_to_name – if True, the string name given in column is the rendered name of the referenced column, not its locally assigned key.
- use_alter – If True, do not emit the DDL for this constraint as part of the CREATE TABLE definition. Instead, generate it via an ALTER TABLE statement issued after the full collection of tables have been created, and drop it via an ALTER TABLE statement before the full collection of tables are dropped. This is shorthand for the usage of AddConstraint and DropConstraint applied as "after-create" and "before-drop" events on the MetaData object. This is normally used to generate/drop constraints on objects that are mutually dependent on each other.
- match – Optional string. If set, emit MATCH <value> when issuing DDL for this constraint. Typical values include SIMPLE, PARTIAL and FULL.
- class sqlalchemy.schema.PrimaryKeyConstraint(*columns, **kw)¶
Bases: sqlalchemy.schema.ColumnCollectionConstraint
A table-level PRIMARY KEY constraint.
Defines a single column or composite PRIMARY KEY constraint. For a no-frills primary key, adding primary_key=True to one or more Column definitions is a shorthand equivalent for an unnamed single- or multiple-column PrimaryKeyConstraint.
- class sqlalchemy.schema.UniqueConstraint(*columns, **kw)¶
Bases: sqlalchemy.schema.ColumnCollectionConstraint
A table-level UNIQUE constraint.
Defines a single column or composite UNIQUE constraint. For a no-frills, single column constraint, adding unique=True to the Column definition is a shorthand equivalent for an unnamed, single column UniqueConstraint.
Indexes can be created anonymously (using an auto-generated name ix_<column label>) for a single column using the inline index keyword on Column, which also modifies the usage of unique to apply the uniqueness to the index itself, instead of adding a separate UNIQUE constraint. For indexes with specific names or which encompass more than one column, use the Index construct, which requires a name.
Below we illustrate a Table with several Index objects associated. The DDL for "CREATE INDEX" is issued right after the create statements for the table:
meta = MetaData()
mytable = Table('mytable', meta,
# an indexed column, with index "ix_mytable_col1"
Column('col1', Integer, index=True),
# a uniquely indexed column with index "ix_mytable_col2"
Column('col2', Integer, index=True, unique=True),
Column('col3', Integer),
Column('col4', Integer),
Column('col5', Integer),
Column('col6', Integer),
# place an index on col3, col4
Index('idx_col34', mytable.c.col3, mytable.c.col4)
# place a unique index on col5, col6
Index('myindex', mytable.c.col5, mytable.c.col6, unique=True)
CREATE TABLE mytable (
col1 INTEGER,
col2 INTEGER,
col3 INTEGER,
col4 INTEGER,
col5 INTEGER,
col6 INTEGER
CREATE INDEX ix_mytable_col1 ON mytable (col1)
CREATE UNIQUE INDEX ix_mytable_col2 ON mytable (col2)
CREATE UNIQUE INDEX myindex ON mytable (col5, col6)
CREATE INDEX idx_col34 ON mytable (col3, col4)
Note in the example above, the Index construct is created externally to the table which it corresponds, using Column objects directly. Index also supports "inline" definition inside the Table, using string names to identify columns:
meta = MetaData()
mytable = Table('mytable', meta,
Column('col1', Integer),
Column('col2', Integer),
Column('col3', Integer),
Column('col4', Integer),
# place an index on col1, col2
Index('idx_col12', 'col1', 'col2'),
# place a unique index on col3, col4
Index('idx_col34', 'col3', 'col4', unique=True)
New in version 0.7: Support of "inline" definition inside the Table for Index.
The Index object also supports its own create() method:
i = Index('someindex', mytable.c.col5)
CREATE INDEX someindex ON mytable (col5)
Functional Indexes¶
Index supports SQL and function expressions, as supported by the target backend. To create an index against a column using a descending value, the ColumnElement.desc() modifier may be used:
from sqlalchemy import Index
Index('someindex', mytable.c.somecol.desc())
Or with a backend that supports functional indexes such as Postgresql, a "case insensitive" index can be created using the lower() function:
from sqlalchemy import func, Index
Index('someindex', func.lower(mytable.c.somecol))
New in version 0.8: Index supports SQL expressions and functions as well as plain columns.
Index API¶
- class sqlalchemy.schema.Index(name, *expressions, **kw)¶
Bases: sqlalchemy.schema.ColumnCollectionMixin, sqlalchemy.schema.SchemaItem
A table-level INDEX.
Defines a composite (one or more column) INDEX. For a no-frills, single column index, adding index=True to the Column definition is a shorthand equivalent for an unnamed, single column Index.
See also
Indexes - General information on Index.
Postgresql-Specific Index Options - PostgreSQL-specific options available for the Index construct.
MySQL Specific Index Options - MySQL-specific options available for the Index construct.
MSSQL-Specific Index Options - MSSQL-specific options available for the Index construct.
- __init__(name, *expressions, **kw)¶
Construct an index object.
Parameters: - name – The name of the index
- *expressions –
Column expressions to include in the index. The expressions are normally instances of Column, but may also be arbitrary SQL expressions which ultmately refer to a Column.
New in version 0.8: Index supports SQL expressions as well as plain columns.
- unique – Defaults to False: create a unique index.
- **kw – Other keyword arguments may be interpreted by specific dialects.
- bind¶
Return the connectable associated with this Index.
- create(bind=None)¶
Issue a CREATE statement for this Index, using the given Connectable for connectivity.
See also MetaData.create_all().
- drop(bind=None)¶
Issue a DROP statement for this Index, using the given Connectable for connectivity.
See also MetaData.drop_all().
Customizing DDL¶
In the preceding sections we've discussed a variety of schema constructs including Table, ForeignKeyConstraint, CheckConstraint, and Sequence. Throughout, we've relied upon the create() and create_all() methods of Table and MetaData in order to issue data definition language (DDL) for all constructs. When issued, a pre-determined order of operations is invoked, and DDL to create each table is created unconditionally including all constraints and other objects associated with it. For more complex scenarios where database-specific DDL is required, SQLAlchemy offers two techniques which can be used to add any DDL based on any condition, either accompanying the standard generation of tables or by itself.
Controlling DDL Sequences¶
The sqlalchemy.schema package contains SQL expression constructs that provide DDL expressions. For example, to produce a CREATE TABLE statement:
from sqlalchemy.schema import CreateTable
CREATE TABLE mytable (
col1 INTEGER,
col2 INTEGER,
col3 INTEGER,
col4 INTEGER,
col5 INTEGER,
col6 INTEGER
Above, the CreateTable construct works like any other expression construct (such as select(), table.insert(), etc.). A full reference of available constructs is in DDL Expression Constructs API.
The DDL constructs all extend a common base class which provides the capability to be associated with an individual Table or MetaData object, to be invoked upon create/drop events. Consider the example of a table which contains a CHECK constraint:
users = Table('users', metadata,
Column('user_id', Integer, primary_key=True),
Column('user_name', String(40), nullable=False),
CheckConstraint('length(user_name) >= 8',name="cst_user_name_length")
CREATE TABLE users (
user_id SERIAL NOT NULL,
user_name VARCHAR(40) NOT NULL,
PRIMARY KEY (user_id),
CONSTRAINT cst_user_name_length CHECK (length(user_name) >= 8)
The above table contains a column "user_name" which is subject to a CHECK constraint that validates that the length of the string is at least eight characters. When a create() is issued for this table, DDL for the CheckConstraint will also be issued inline within the table definition.
The CheckConstraint construct can also be constructed externally and associated with the Table afterwards:
constraint = CheckConstraint('length(user_name) >= 8',name="cst_user_name_length")
So far, the effect is the same. However, if we create DDL elements corresponding to the creation and removal of this constraint, and associate them with the Table as events, these new events will take over the job of issuing DDL for the constraint. Additionally, the constraint will be added via ALTER:
from sqlalchemy import event
CREATE TABLE users (
user_id SERIAL NOT NULL,
user_name VARCHAR(40) NOT NULL,
PRIMARY KEY (user_id)
ALTER TABLE users ADD CONSTRAINT cst_user_name_length CHECK (length(user_name) >= 8)
ALTER TABLE users DROP CONSTRAINT cst_user_name_length
DROP TABLE user
The real usefulness of the above becomes clearer once we illustrate the DDLElement.execute_if() method. This method returns a modified form of the DDL callable which will filter on criteria before responding to a received event. It accepts a parameter dialect, which is the string name of a dialect or a tuple of such, which will limit the execution of the item to just those dialects. It also accepts a callable_ parameter which may reference a Python callable which will be invoked upon event reception, returning True or False indicating if the event should proceed.
If our CheckConstraint was only supported by Postgresql and not other databases, we could limit its usage to just that dialect:
Or to any set of dialects:
AddConstraint(constraint).execute_if(dialect=('postgresql', 'mysql'))
DropConstraint(constraint).execute_if(dialect=('postgresql', 'mysql'))
When using a callable, the callable is passed the ddl element, the Table or MetaData object whose "create" or "drop" event is in progress, and the Connection object being used for the operation, as well as additional information as keyword arguments. The callable can perform checks, such as whether or not a given item already exists. Below we define should_create() and should_drop() callables that check for the presence of our named constraint:
def should_create(ddl, target, connection, **kw):
row = connection.execute("select conname from pg_constraint where conname='%s'" % ddl.element.name).scalar()
return not bool(row)
def should_drop(ddl, target, connection, **kw):
return not should_create(ddl, target, connection, **kw)
CREATE TABLE users (
user_id SERIAL NOT NULL,
user_name VARCHAR(40) NOT NULL,
PRIMARY KEY (user_id)
select conname from pg_constraint where conname='cst_user_name_length'
ALTER TABLE users ADD CONSTRAINT cst_user_name_length CHECK (length(user_name) >= 8)
select conname from pg_constraint where conname='cst_user_name_length'
ALTER TABLE users DROP CONSTRAINT cst_user_name_length
DROP TABLE user
Custom DDL¶
Custom DDL phrases are most easily achieved using the DDL construct. This construct works like all the other DDL elements except it accepts a string which is the text to be emitted:
DDL("ALTER TABLE users ADD CONSTRAINT "
"cst_user_name_length "
" CHECK (length(user_name) >= 8)")
A more comprehensive method of creating libraries of DDL constructs is to use custom compilation - see Custom SQL Constructs and Compilation Extension for details.
DDL Expression Constructs API¶
- class sqlalchemy.schema.DDLElement¶
Bases: sqlalchemy.sql.expression.Executable, sqlalchemy.schema._DDLCompiles
Base class for DDL expression constructs.
This class is the base for the general purpose DDL class, as well as the various create/drop clause constructs such as CreateTable, DropTable, AddConstraint, etc.
DDLElement integrates closely with SQLAlchemy events, introduced in Events. An instance of one is itself an event receiving callable:
event.listen( users, 'after_create', AddConstraint(constraint).execute_if(dialect='postgresql') )
See also:
- __call__(target, bind, **kw)¶
Execute the DDL as a ddl_listener.
- against(target)¶
Return a copy of this DDL against a specific schema item.
- bind¶
- callable_ = None¶
- dialect = None¶
- execute(bind=None, target=None)¶
Execute this DDL immediately.
Executes the DDL statement in isolation using the supplied Connectable or Connectable assigned to the .bind property, if not supplied. If the DDL has a conditional on criteria, it will be invoked with None as the event.
Parameters: - bind – Optional, an Engine or Connection. If not supplied, a valid Connectable must be present in the .bind property.
- target – Optional, defaults to None. The target SchemaItem for the execute call. Will be passed to the on callable if any, and may also provide string expansion data for the statement. See execute_at for more information.
- execute_at(event_name, target)¶
Link execution of this DDL to the DDL lifecycle of a SchemaItem.
Deprecated since version 0.7: See DDLEvents, as well as DDLElement.execute_if().
Links this DDLElement to a Table or MetaData instance, executing it when that schema item is created or dropped. The DDL statement will be executed using the same Connection and transactional context as the Table create/drop itself. The .bind property of this statement is ignored.
Parameters: - event – One of the events defined in the schema item's .ddl_events; e.g. 'before-create', 'after-create', 'before-drop' or 'after-drop'
- target – The Table or MetaData instance for which this DDLElement will be associated with.
A DDLElement instance can be linked to any number of schema items.
execute_at builds on the append_ddl_listener interface of MetaData and Table objects.
Caveat: Creating or dropping a Table in isolation will also trigger any DDL set to execute_at that Table's MetaData. This may change in a future release.
- execute_if(dialect=None, callable_=None, state=None)¶
Return a callable that will execute this DDLElement conditionally.
Used to provide a wrapper for event listening:
event.listen( metadata, 'before_create', DDL("my_ddl").execute_if(dialect='postgresql') )
Parameters: - dialect –
May be a string, tuple or a callable predicate. If a string, it will be compared to the name of the executing database dialect:
If a tuple, specifies multiple dialect names:
DDL('something').execute_if(dialect=('postgresql', 'mysql'))
- callable –
A callable, which will be invoked with four positional arguments as well as optional keyword arguments:
ddl: This DDL element. target: The Table or MetaData object which is the target of this event. May be None if the DDL is executed explicitly. bind: The Connection being used for DDL execution tables: Optional keyword argument - a list of Table objects which are to be created/ dropped within a MetaData.create_all() or drop_all() method call. state: Optional keyword argument - will be the state argument passed to this function. checkfirst: Keyword argument, will be True if the 'checkfirst' flag was set during the call to create(), create_all(), drop(), drop_all(). If the callable returns a true value, the DDL statement will be executed.
- state – any value which will be passed to the callable_ as the state keyword argument.
See also:
- dialect –
- on = None¶
- target = None¶
- class sqlalchemy.schema.DDL(statement, on=None, context=None, bind=None)¶
Bases: sqlalchemy.schema.DDLElement
A literal DDL statement.
Specifies literal SQL DDL to be executed by the database. DDL objects function as DDL event listeners, and can be subscribed to those events listed in DDLEvents, using either Table or MetaData objects as targets. Basic templating support allows a single DDL instance to handle repetitive tasks for multiple tables.
from sqlalchemy import event, DDL tbl = Table('users', metadata, Column('uid', Integer)) event.listen(tbl, 'before_create', DDL('DROP TRIGGER users_trigger')) spow = DDL('ALTER TABLE %(table)s SET secretpowers TRUE') event.listen(tbl, 'after_create', spow.execute_if(dialect='somedb')) drop_spow = DDL('ALTER TABLE users SET secretpowers FALSE') connection.execute(drop_spow)
When operating on Table events, the following statement string substitions are available:
%(table)s - the Table name, with any required quoting applied %(schema)s - the schema name, with any required quoting applied %(fullname)s - the Table name including schema, quoted if needed
The DDL's "context", if any, will be combined with the standard substutions noted above. Keys present in the context will override the standard substitutions.
- __init__(statement, on=None, context=None, bind=None)¶
Create a DDL statement.
Parameters: - statement –
A string or unicode string to be executed. Statements will be processed with Python's string formatting operator. See the context argument and the execute_at method.
A literal '%' in a statement must be escaped as '%%'.
SQL bind parameters are not available in DDL statements.
- on –
Deprecated. See DDLElement.execute_if().
Optional filtering criteria. May be a string, tuple or a callable predicate. If a string, it will be compared to the name of the executing database dialect:
DDL('something', on='postgresql')
If a tuple, specifies multiple dialect names:
DDL('something', on=('postgresql', 'mysql'))
If a callable, it will be invoked with four positional arguments as well as optional keyword arguments:
ddl: This DDL element. event: The name of the event that has triggered this DDL, such as 'after-create' Will be None if the DDL is executed explicitly. target: The Table or MetaData object which is the target of this event. May be None if the DDL is executed explicitly. connection: The Connection being used for DDL execution tables: Optional keyword argument - a list of Table objects which are to be created/ dropped within a MetaData.create_all() or drop_all() method call. If the callable returns a true value, the DDL statement will be executed.
- context – Optional dictionary, defaults to None. These values will be available for use in string substitutions on the DDL statement.
- bind – Optional. A Connectable, used by default when execute() is invoked without a bind argument.
See also:
DDLEvents sqlalchemy.event- statement –
- class sqlalchemy.schema.CreateTable(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a CREATE TABLE statement.
- __init__(element, on=None, bind=None)¶
Create a CreateTable construct.
- class sqlalchemy.schema.DropTable(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a DROP TABLE statement.
- class sqlalchemy.schema.CreateColumn(element)¶
Bases: sqlalchemy.schema._DDLCompiles
Represent a Column as rendered in a CREATE TABLE statement, via the CreateTable construct.
This is provided to support custom column DDL within the generation of CREATE TABLE statements, by using the compiler extension documented in Custom SQL Constructs and Compilation Extension to extend CreateColumn.
Typical integration is to examine the incoming Column object, and to redirect compilation if a particular flag or condition is found:
from sqlalchemy import schema from sqlalchemy.ext.compiler import compiles @compiles(schema.CreateColumn) def compile(element, compiler, **kw): column = element.element if "special" not in column.info: return compiler.visit_create_column(element, **kw) text = "%s SPECIAL DIRECTIVE %s" % ( column.name, compiler.type_compiler.process(column.type) ) default = compiler.get_column_default_string(column) if default is not None: text += " DEFAULT " + default if not column.nullable: text += " NOT NULL" if column.constraints: text += " ".join( compiler.process(const) for const in column.constraints) return text
The above construct can be applied to a Table as follows:
from sqlalchemy import Table, Metadata, Column, Integer, String from sqlalchemy import schema metadata = MetaData() table = Table('mytable', MetaData(), Column('x', Integer, info={"special":True}, primary_key=True), Column('y', String(50)), Column('z', String(20), info={"special":True}) ) metadata.create_all(conn)
Above, the directives we've added to the Column.info collection will be detected by our custom compilation scheme:
CREATE TABLE mytable ( x SPECIAL DIRECTIVE INTEGER NOT NULL, y VARCHAR(50), z SPECIAL DIRECTIVE VARCHAR(20), PRIMARY KEY (x) )
New in version 0.8: The CreateColumn construct was added to support custom column creation styles.
- class sqlalchemy.schema.CreateSequence(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a CREATE SEQUENCE statement.
- class sqlalchemy.schema.DropSequence(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a DROP SEQUENCE statement.
- class sqlalchemy.schema.CreateIndex(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a CREATE INDEX statement.
- class sqlalchemy.schema.DropIndex(element, on=None, bind=None)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a DROP INDEX statement.
- class sqlalchemy.schema.AddConstraint(element, *args, **kw)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent an ALTER TABLE ADD CONSTRAINT statement.
- class sqlalchemy.schema.DropConstraint(element, cascade=False, **kw)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent an ALTER TABLE DROP CONSTRAINT statement.
- class sqlalchemy.schema.CreateSchema(name, quote=None, **kw)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a CREATE SCHEMA statement.
New in version 0.7.4.
The argument here is the string name of the schema.
- __init__(name, quote=None, **kw)¶
Create a new CreateSchema construct.
- class sqlalchemy.schema.DropSchema(name, quote=None, cascade=False, **kw)¶
Bases: sqlalchemy.schema._CreateDropBase
Represent a DROP SCHEMA statement.
The argument here is the string name of the schema.
New in version 0.7.4.
- __init__(name, quote=None, cascade=False, **kw)¶
Create a new DropSchema construct.