SQL Expressions

    The “stringification” of a SQLAlchemy Core statement object or expression fragment, as well as that of an ORM object, in the majority of simple cases is as simple as using the str() builtin function, as below when use it with the print function (note the Python print function also calls str() automatically if we don’t use it explicitly):

    The str() builtin, or an equivalent, can be invoked on ORM Query object as well as any statement such as that of , insert() etc. and also any expression fragment, such as:

    1. >>> from sqlalchemy import column
    2. >>> print(column('x') == 'some value')
    3. x = :x_1

    A complication arises when the statement or fragment we are stringifying contains elements that have a database-specific string format, or when it contains elements that are only available within a certain kind of database. In these cases, we might get a stringified statement that is not in the correct syntax for the database we are targeting, or the operation may raise a exception. In these cases, it is necessary that we stringify the statement using the ClauseElement.compile() method, while passing along an or Dialect object that represents the target database. Such as below, if we have a MySQL database engine, we can stringify a statement in terms of the MySQL dialect:

    1. from sqlalchemy import create_engine
    3. engine = create_engine("mysql+pymysql://scott:tiger@localhost/test")
    4. print(statement.compile(engine))

    More directly, without building up an object we can instantiate a Dialect object directly, as below where we use a PostgreSQL dialect:

    1. from sqlalchemy.dialects import postgresql
    2. print(statement.compile(dialect=postgresql.dialect()))

    When given an ORM Query object, in order to get at the method we only need access the Query.statement accessor first:

    1. statement = query.statement
    2. print(statement.compile(someengine))

    Warning

    Never use this technique with string content received from untrusted input, such as from web forms or other user-input applications. SQLAlchemy’s facilities to coerce Python values into direct SQL string values are not secure against untrusted input and do not validate the type of data being passed. Always use bound parameters when programmatically invoking non-DDL SQL statements against a relational database.

    1. from sqlalchemy.sql import table, column, select
    2. t = table('t', column('x'))
    4. s = select(t).where(t.c.x == 5)
    6. # **do not use** with untrusted input!!!
    7. print(s.compile(compile_kwargs={"literal_binds": True}))

    The above approach has the caveats that it is only supported for basic types, such as ints and strings, and furthermore if a bindparam() without a pre-set value is used directly, it won’t be able to stringify that either.

    This functionality is provided mainly for logging or debugging purposes, where having the raw sql string of a query may prove useful. Note that the dialect parameter should also passed to the method to render the query that will be sent to the database.

    To support inline literal rendering for types not supported, implement a TypeDecorator for the target type which includes a method:

    1. from sqlalchemy import TypeDecorator, Integer
    4. class MyFancyType(TypeDecorator):
    5.     impl = Integer
    7.     def process_literal_param(self, value, dialect):
    8.         return "my_fancy_formatting(%s)" % value
    10. from sqlalchemy import Table, Column, MetaData
    12. tab = Table('mytable', MetaData(), Column('x', MyFancyType()))
    14. stmt = tab.select().where(tab.c.x > 5)
    15. print(stmt.compile(compile_kwargs={"literal_binds": True}))

    producing output like:

    Many DBAPI implementations make use of the pyformat or format , which necessarily involve percent signs in their syntax. Most DBAPIs that do this expect percent signs used for other reasons to be doubled up (i.e. escaped) in the string form of the statements used, e.g.:

    1. SELECT a, b FROM some_table WHERE a = %s AND c = %s AND num %% modulus = 0

    When SQL statements are passed to the underlying DBAPI by SQLAlchemy, substitution of bound parameters works in the same way as the Python string interpolation operator %, and in many cases the DBAPI actually uses this operator directly. Above, the substitution of bound parameters would then look like:

    1. SELECT a, b FROM some_table WHERE a = 5 AND c = 10 AND num % modulus = 0

    The default compilers for databases like PostgreSQL (default DBAPI is psycopg2) and MySQL (default DBAPI is mysqlclient) will have this percent sign escaping behavior:

    1. >>> from sqlalchemy import table, column
    2. >>> t = table("my_table", column("value % one"), column("value % two"))
    3. >>> print(t.select().compile(dialect=postgresql.dialect()))
    5. FROM my_table

    When such a dialect is being used, if non-DBAPI statements are desired that don’t include bound parameter symbols, one quick way to remove the percent signs is to simply substitute in an empty set of parameters using Python’s % operator directly:

    1. >>> strstmt = str(t.select().compile(dialect=postgresql.dialect()))
    2. >>> print(strstmt % ())
    3. SELECT my_table."value % one", my_table."value % two"
    4. FROM my_table

    The other is to set a different parameter style on the dialect being used; all Dialect implementations accept a parameter paramstyle which will cause the compiler for that dialect to use the given parameter style. Below, the very common named parameter style is set within the dialect used for the compilation so that percent signs are no longer significant in the compiled form of SQL, and will no longer be escaped:

    1. >>> print(t.select().compile(dialect=postgresql.dialect(paramstyle="named")))
    2. SELECT my_table."value % one", my_table."value % two"
    3. FROM my_table

    The method allows one to create a custom database operator otherwise not known by SQLAlchemy:

    1. >>> print(column('q').op('->')(column('p')))
    2. q -> p

    However, when using it on the right side of a compound expression, it doesn’t generate parenthesis as we expect:

    The solution to this case is to set the precedence of the operator, using the Operators.op.precedence parameter, to a high number, where 100 is the maximum value, and the highest number used by any SQLAlchemy operator is currently 15:

    1. >>> print((column('q1') + column('q2')).op('->', precedence=100)(column('p')))
    2. (q1 + q2) -> p

    We can also usually force parenthesization around a binary expression (e.g. an expression that has left/right operands and an operator) using the method:

    1. >>> print((column('q1') + column('q2')).self_group().op('->')(column('p')))
    2. (q1 + q2) -> p

    A lot of databases barf when there are excessive parenthesis or when parenthesis are in unusual places they doesn’t expect, so SQLAlchemy does not generate parenthesis based on groupings, it uses operator precedence and if the operator is known to be associative, so that parenthesis are generated minimally. Otherwise, an expression like:

    1. column('a') & column('b') & column('c') & column('d')

    would produce:

    1. (((a AND b) AND c) AND d)

    which is fine but would probably annoy people (and be reported as a bug). In other cases, it leads to things that are more likely to confuse databases or at the very least readability, such as:

    1. column('q', ARRAY(Integer, dimensions=2))[5][6]

    would produce:

    1. ((q[5])[6])

    There are also some edge cases where we get things like "(x) = 7" and databases really don’t like that either. So parenthesization doesn’t naively parenthesize, it uses operator precedence and associativity to determine groupings.

    For Operators.op(), the value of precedence defaults to zero.

    What if we defaulted the value of to 100, e.g. the highest? Then this expression makes more parenthesis, but is otherwise OK, that is, these two are equivalent:

    but these two are not:

    1. >>> print(column('q') - column('y').op('+', precedence=100)(column('z')))
    2. q - y + z
    3. >>> print(column('q') - column('y').op('+')(column('z')))

    For now, it’s not clear that as long as we are doing parenthesization based on operator precedence and associativity, if there is really a way to parenthesize automatically for a generic operator with no precedence given that is going to work in all cases, because sometimes you want a custom op to have a lower precedence than the other operators and sometimes you want it to be higher.