mlm_insights.core.transformers.asteval package

Submodules

mlm_insights.core.transformers.asteval.ast_evaluator module

Safe(ish) evaluation of mathematical expression using Python’s ast module.

This module provides an Interpreter class that compiles a restricted set of Python expressions and statements to Python’s AST representation, and then executes that representation using values held in a symbol table.

The symbol table is a simple dictionary, giving a simple, flat namespace. This comes pre-loaded with many functions from Python’s builtin and math module. If numpy is installed, many numpy functions are also included. Additional symbols can be added when an Interpreter is created, but the user of that interpreter will not be able to import additional modules.

Expressions, including loops, conditionals, and function definitions can be compiled into ast node and then evaluated later, using the current values in the symbol table.

The result is a restricted, simplified version of Python meant for numerical calculations that is somewhat safer than ‘eval’ because many unsafe operations (such as ‘import’ and ‘eval’) are simply not allowed.

Many parts of Python syntax are supported, including:

for loops, while loops, if-then-elif-else conditionals try-except (including ‘finally’) function definitions with def advanced slicing: a[::-1], array[-3:, :, ::2] if-expressions: out = one_thing if TEST else other list comprehension out = [sqrt(i) for i in values]

The following Python syntax elements are not supported:

Import, Exec, Lambda, Class, Global, Generators, Yield, Decorators

In addition, while many builtin functions are supported, several builtin functions that are considered unsafe are missing (‘eval’, ‘exec’, and ‘getattr’ for example)

class mlm_insights.core.transformers.asteval.ast_evaluator.Empty

Bases: object

Empty class.

class mlm_insights.core.transformers.asteval.ast_evaluator.ExceptionHolder(node, exc=None, msg='', expr=None, lineno=None)

Bases: object

Basic exception handler.

get_error()

Retrieve error data.

class mlm_insights.core.transformers.asteval.ast_evaluator.Interpreter(symtable=None, usersyms=None, writer=None, err_writer=None, use_numpy=True, max_statement_length=50000, minimal=False, no_if=False, no_for=False, no_while=False, no_try=False, no_functiondef=False, no_ifexp=False, no_listcomp=False, no_augassign=False, no_assert=False, no_delete=False, no_raise=False, no_print=False, max_time=None, readonly_symbols=None, builtins_readonly=False, ast_parse_mode='eval')

Bases: object

create an asteval Interpreter: a restricted, simplified interpreter of mathematical expressions using Python syntax.

Parameters

symtabledict or None

dictionary to use as symbol table (if None, one will be created).

usersymsdict or None

dictionary of user-defined symbols to add to symbol table.

writerfile-like or None

callable file-like object where standard output will be sent.

err_writerfile-like or None

callable file-like object where standard error will be sent.

use_numpybool

whether to use functions from numpy.

max_statement_lengthint

maximum length of expression allowed [50,000 characters]

minimalbool

create a minimal interpreter: disable all options (see Note 1).

no_ifbool

whether to support if blocks

no_forbool

whether to support for blocks.

no_whilebool

whether to support while blocks.

no_trybool

whether to support try blocks.

no_functiondefbool

whether to support user-defined functions.

no_ifexpbool

whether to support if expressions.

no_listcompbool

whether to support list comprehension.

no_augassignbool

whether to support augemented assignments (a += 1, etc).

no_assertbool

whether to support assert.

no_deletebool

whether to support del.

no_raisebool

whether to support raise.

no_printbool

whether to support print.

readonly_symbolsiterable or None

symbols that the user can not assign to

builtins_readonlybool

whether to blacklist all symbols that are in the initial symtable

Notes

1. setting minimal=True is equivalent to setting all no_*** options to True.

static dump(node, **kw)

Simple ast dumper.

eval(expr, lineno=0, show_errors=True, raise_errors=False)

Evaluate a single statement.

node_assign(node, val)

Assign a value (not the node.value object) to a node.

This is used by on_assign, but also by for, list comprehension, etc.

on_arg(node)

Arg for function definitions.

on_assert(node)

Assert statement.

on_assign(node)

Simple assignment.

on_attribute(node)

Extract attribute.

on_augassign(node)

Augmented assign.

on_binop(node)

Binary operator.

on_boolop(node)

Boolean operator.

on_break(node)

Break.

on_call(node)

Function execution.

on_compare(node)

comparison operators, including chained comparisons (a<b<c)

on_constant(node)

Return constant value.

on_continue(node)

Continue.

on_delete(node)

Delete statement.

on_dict(node)

Dictionary.

on_ellipsis(node)

Ellipses.

on_excepthandler(node)

Exception handler…

on_expr(node)

Expression.

on_expression(node)

basic expression

on_extslice(node)

Extended slice.

on_for(node)

For blocks.

on_functiondef(node)

Define procedures.

on_if(node)

Regular if-then-else statement.

on_ifexp(node)

If expressions.

on_index(node)

Index.

on_interrupt(node)

Interrupt handler.

on_list(node)

List.

on_listcomp(node)

List comprehension – only up to 4 generators!

on_module(node)

Module def.

on_name(node)

Name node.

on_nameconstant(node)

True, False, or None

on_num(node)

Return number.

on_pass(node)

Pass statement.

on_raise(node)

Raise statement: note difference for python 2 and 3.

on_repr(node)

Repr.

on_return(node)

Return statement: look for None, return special sentinel.

on_slice(node)

Simple slice.

on_str(node)

Return string.

on_subscript(node)

Subscript handling – one of the tricky parts.

on_try(node)

Try/except/else/finally blocks.

on_tuple(node)

Tuple.

on_unaryop(node)

Unary operator.

on_while(node)

While blocks.

parse(text)

Parse statement/expression to Ast representation.

raise_exception(node, exc=None, msg='', expr=None, lineno=None)

Add an exception.

remove_nodehandler(node)

remove support for a node returns current node handler, so that it might be re-added with add_nodehandler()

run(node, expr=None, lineno=None, with_raise=True)

Execute parsed Ast representation for an expression.

set_nodehandler(node, handler)

set node handler

unimplemented(node)

Unimplemented nodes.

user_defined_symbols()

Return a set of symbols that have been added to symtable after construction.

I.e., the symbols from self.symtable that are not in self.no_deepcopy.

Returns

unique_symbolsset

symbols in symtable that are not in self.no_deepcopy

mlm_insights.core.transformers.asteval.ast_evaluator.NAME_MATCH(string, pos=0, endpos=9223372036854775807)

Matches zero or more characters at the beginning of the string.

class mlm_insights.core.transformers.asteval.ast_evaluator.NameFinder

Bases: NodeVisitor

Find all symbol names used by a parsed node.

generic_visit(node)

TODO: docstring in public method.

class mlm_insights.core.transformers.asteval.ast_evaluator.Procedure(name, interp, doc=None, lineno=0, body=None, args=None, kwargs=None, vararg=None, varkws=None)

Bases: object

Procedure: user-defined function for asteval.

This stores the parsed ast nodes as from the ‘functiondef’ ast node for later evaluation.

mlm_insights.core.transformers.asteval.ast_evaluator.get_ast_names(astnode)

Return symbol Names from an AST node.

mlm_insights.core.transformers.asteval.ast_evaluator.make_symbol_table(use_numpy=True, **kws)

Create a default symboltable, taking dict of user-defined symbols.

Arguments

numpybool, optional

whether to include symbols from numpy

kwsoptional

additional symbol name, value pairs to include in symbol table

Returns

symbol_tabledict

a symbol table that can be used in asteval.Interpereter

mlm_insights.core.transformers.asteval.ast_evaluator.op2func(op)

Return function for operator nodes.

mlm_insights.core.transformers.asteval.ast_evaluator.safe_add(a, b)

safe version of add

mlm_insights.core.transformers.asteval.ast_evaluator.safe_lshift(a, b)

safe version of lshift

mlm_insights.core.transformers.asteval.ast_evaluator.safe_mult(a, b)

safe version of multiply

mlm_insights.core.transformers.asteval.ast_evaluator.safe_pow(base, exp)

safe version of pow

mlm_insights.core.transformers.asteval.ast_evaluator.valid_symbol_name(name)

Determine whether the input symbol name is a valid name.

Arguments

namestr

name to check for validity.

Returns

validbool

whether name is a a valid symbol name

This checks for Python reserved words and that the name matches the regular expression [a-zA-Z_][a-zA-Z0-9_]

Module contents