							Save your money via IT outsourcing!
View source

From OpenOpt

for FuncDesignerDoc
Jump to: navigation, search
You do not have permission to edit pages, for the following reason:
The action you have requested is limited to users in the group Sysops.
You can view and copy the source of this page:
<font color=blue size=5><center>'''FuncDesigner Documentation'''</center></font> 


__TOC__

= Creating oovars =
oovars are Object-Oriented Variables (initially, when FuncDesigner was a part of OpenOpt module - OpenOpt variables).
Their creating is performed via oovar() and oovars(). Let's note - each oovar and oofun has field "name", that is used in error messages and for some parts of FuncDesigner kernel. Names should be unique. 
<source lang = "python">
from FuncDesigner import *
a = oovar() # create oovar a
b = oovar('b') # create oovar b named "b" instead of something like "unnamed_34"
a, b, c = oovars('a', 'b', 'c') # create 3 oovars named "a", "b", "c", each one can have different size
a, b, c, d = oovars(4) # create 4 oovars, each one can have different size

myOOVars = oovars(80) # create Python list of 80 oovars, each one can have different size
# Beware! - since myOOVars is Python list, math operations differs from array-like: 
# 2*myOOVars will be list of length 160, myOOVars + someOtherOOVars will be concatenated list etc
# You should perform operations like listOfDoubledValues = [2*myOOVars[i] for i in xrange(len(myOOVars))]
</source>

Also you can define general n-dimensional numpy arrays of oovars (requires good knowledge of numpy masked arrays), each one of different size. However, for the sake of calculations speed it is recommended to create lots of oovars with size 1, as well as a small number of oovars with huge sizes and involving lots of slicing/indexation operations (like x[i], x[i:j]).

Since FD 0.17, if you want to raise a error when in a point involved size of oovar differs from the expected one, you could use parameter "size" in oovar constructor, e.g.
<source lang = "python">
a = oovar(size=some_expected_size) 
# or with user-provided name
a = oovar('a', size=4) 
</source>


= Creating oofuns =
Functions applied on oovars produce oofuns, eg
<source lang = python>
from FuncDesigner import *
a, b, c, d = oovars(4) 
f1, f2 = a + 2*b + c, sin(a)+cos(b)+arctan(c)
f3 = f1 + 4*f2 + cos(f1*f2)
# you can involve in-place operations:
f3 += sin(f2)+cos(f3)
f3 /= 1024
f3 **= 2 
</source>
To obtain value of oofun we should define a point that is a [[PythonIntroduction#Dictionary|Python dictionary]] with keys oovars and values = their values, e.g.
<source lang = python>
from numpy import *
point1 = {a:1, b:[2,3,4], c:array([10,20,30])}
r = f3(point1)
print(r) # It should print [ 21.60063885  33.28742147  45.64660797]
</source>
Some notes:
* Python lists (like the one above with value [2,3,4]) are simpler to type but are not recommended - one can forget to cast them into arrays, involve [1,2] + [5,10] and obtain [1,2,5,10] instead of [6, 12]. In openopt, scipy, lots of other Python scientific packages lists are cast to numpy arrays, yet one cannot be sure there are no any bugs lurking. So, creating Python lists of oovars (eg a=oovars(N)) is not recommended as well unless you exactly know what are you doing.<br>
*{{bug}}Current FuncDesigner release 0.17 has the bug: you should avoid operations with right-side oofun and left-side array, e.g. a*f, a/f, a**f, f**a, a-f, a+f etc (a is array and f is oofun or oovar). It will produce numpy array of size same to size(a). Here you should use Python lists instead: a.tolist()*f,a.tolist()/f etc or operations like f*a, f**(-1)*a, -f+a, f+a etc. The issue is fixed in subversion repository, so you can update it from svn or zip-file from [[Install|installation webpage]].
* You can use summation of Python lists made of oovars/oofuns/numbers/arrays eg 
<source lang = python>
datalist = [a, b, some_oofun_1, 2.0, N, 4, some_oofun_2] # (*)
# or
datalist = [a, [1,2,3], some_oofun_3] # (**)
c = sum(datalist) 
</source>
:provided all elements under sum() have same size (1 for (*) and 3 for (**)). 
* '''You can use "for" cycle''', eg <source lang = "python">M = 15; for i in xrange(M): f3 = 0.1*f3+0.2*f2+1</source>
* We can use "while" cycle as well, provided arguments in '''"for"/"while"''' loops are independent on any oovar/oofun.

To name an oofun (for some error messages and/or debug purposes) you can use one of the ways below:
<source lang = "python">
K = 0.5*m*V**2; K.name = 'kinetic energy'; P = g*h*m; P.name = 'potential energy'; TotalEnergy = K + P
TotalEnergy = (0.5*m*V**2).named('kinetic energy') + (m*g*h).named('potential energy') 
</source>
Each oofun should return a single number or array with ndim <= 1; oovars also should be numbers or arrays with ndim <= 1.<br>
Currently only basic numpy operators and functions have been overloaded, if you have something exotic like gamma-function or arctan2 you should create your own oofun (and provide analytical derivative for that one, if it is possible/has sense), see further doc chapters.


= Automatic differentiation =
We can get oofun derivative in a point via '''[[AD|automatic differentiation]]''' (not to be confused with [http://en.wikipedia.org/wiki/Numerical_differentiation Numerical differentiation] via [http://en.wikipedia.org/wiki/Finite_difference finite-differences derivatives approximation] and symbolic differentiation provided by [http://maxima.sourceforge.net/ Maxima], [http://code.google.com/p/sympy/ SymPy] etc).
<source lang = "python">
from FuncDesigner import *
a, b, c = oovars('a', 'b', 'c')
f1, f2 = sin(a) + cos(b) - log2(c) + sqrt(b), sum(c) + c * cosh(b) / arctan(a) + c[0] * c[1] + c[-1] / (a * c.size)
f3 = f1*f2 + 2*a + sin(b) * (1+2*c.size + 3*f2.size)
f = (2*a*b*c + f1*f2 + f3 + dot(a+c, b+c))


point = {a:1, b:2, c:[3, 4, 5]} # however, you'd better use numpy arrays instead of Python lists

print(f(point))
print(f.D(point))
print(f.D(point, a))
print(f.D(point, [b]))
print(f.D(point, fixedVars = [a, c]))

M = [[1, 2, 3], [2, 3, 4], [2, 5, 9]] # 2D matrix

_ff = f +  dot(M, sinh(a) + c + c.size + a.size) 
ff = 2*cos(_ff) + 3*sin(f)
print(ff.D(point))

""" Expected output: 
[ 140.9337138   110.16255336   80.67870244]
{a: array([  69.75779959,   88.89020412,  109.93551537]), b: array([-23.10565554, -39.41138045, -59.08378522]), 
c: array([[-20.52297003,  13.03660168,  13.67886723],
       [  7.39537711, -20.15709726,  12.57210056],
       [  4.17609616,   7.14087693, -17.54104071]])}
[  69.75779959   88.89020412  109.93551537]
{b: array([-23.10565554, -39.41138045, -59.08378522])}
{b: array([-23.10565554, -39.41138045, -59.08378522])}
{a: array([  1.19506477,   1.09352624, -44.81591112]), b: array([  59.79244279,  112.61139818,   11.88339733]), 
c: array([[  0.12907781,   0.25815561,   0.38723342],
       [  0.15748097,   0.23622145,   0.31496193],
       [ -3.63039284,  -9.0759821 , -16.33676778]])}
"""
</source>


= Operator ifThenElse =
{{new}} ([[coming]]) '''You can use operator "ifThenElse"''' where ordinary Python language operator if-then-else can't deal with condition because it has type ''oofun'' (that hasn't been invoked on a point yet and thus preliminary unknown) instead of ''boolean'', e.g.
<source lang=python>
from FuncDesigner import *
a, b = oovars('a', 'b')
point = {a:10, b:[2, -5, -4]}
f1 = ifThenElse(a+sum(b)<1, 3*a, 15*(a+5/b[0]))
f2 = 4*sin(a) + 5 * ifThenElse(a+sum(abs(b))>1, 3, 4*a)
f3 = 15 + 25 * ifThenElse(2>1, -100, 3*sum(log2(abs(b))))
f4 = 15 + 25 * ifThenElse(1>2, -100, 3*sum(log2(abs(b))))
for f in [f1, f2, f3, f4]: 
    print(f(point))
    print(f.D(point))

#[ 187.5]
#{a: array([ 15.]), b: array([-18.75,  -0.  ,  -0.  ])}
#[ 12.82391556]
#{a: array([-3.35628612])}
#[-2485]
#{}
#[ 414.14460712]
#{b: array([ 54.10106403, -21.64042561, -27.05053202])}
</source>
If your condition has type boolean when Python loads it (for example when you have "if myOOFun(startPoint)"), then you can use ordinary python if-elif-...-then-else in spite of type of other data involved.

= Passing oovars through ordinary Python functions =
oovars can pass through ordinary Python functions <br>
provided FuncDesigner has appropriate overloads for all functions/operators<br>
used inside those ones

<source lang=python>
from FuncDesigner import *
a, b, c = oovars('a', 'b', 'c')

def func1(x, y):
    return x + 4*y
    
func2 = lambda x, y, z: sin(x) + 4*y + 3*cos(z) + func1(x, z)
    
def func3(x, y, z):
    return x + 2*y + 5*sin(z) + func2(x, z, y) + func1(z, y)

print(func3(2, 3, 4))

# Note that func3, func2, func1 could be imported from another file instead of been defined here
myFunc = func3(a, b, c) # now myFunc is instance of FuncDesigner class oofun
point = {a:2, b:3, c:4} 

print(myFunc(point))
print('difference: %f'% (func3(2, 3, 4) - myFunc(point)))
print('Derivative obtained via Automatic differentiation:')
print(myFunc.D(point))

# Expected output:
# 48.1553074605
# [ 48.15530746]
# difference: 0.000000
# Derivative obtained via Automatic differentiation:
# {a: array([ 1.58385316]), b: array([ 9.57663998]), c: array([ 1.7317819])}
</source>


= Solving systems of linear equations =
(requires FD ver >= 0.17, OO ver >= 0.27)<br>
Currently 
* it requires OpenOpt installed, maybe in future I'll get rid of the dependence
* available solvers is numpy.linalg.solve (LAPACK routine dgesv, dense systems only) and scipy.sparse.linalg.spsolve (LAPACK routine dgessv for sparse systems). Also, you can use your own solver (see example with rendering below).
* Note - time for solving may differ for same SLEs due to different matrices obtained from rendering, because Python 2.6 doesn't has ordered sets (they are present in Python 3.x). Maybe I'll implement fixed rendering in future for 3.x, I don't want to deal with quite difficult walkaround for v. 2.6.
{{FD_SLE}}


= Creating special oofuns =
If you have functions beyond current FuncDesigner overloads capabilities and/or want to provide oofun wrappers for funcs written in C, Fortran, other languages, you can create your own oofun(s).<br>
[http://openopt.org/FuncDesignerDoc#Automatic_differentiation Automatic differentiation] will be available provided you have [[DerApproximator]] installed (it will substitute finite-differences approximation for oofuns w/o user-supplied derivatives).
Generally creating special oofuns is performed via
<source lang=python>
from FuncDesigner import oofun
my_oofun = oofun(function, input=input, (optional)d=d, args=args)
</source>
where function is Python function (or connected to Python by f2py, Cython, ctypes etc),<br>
input is oovar(s)/oofun(s) (list or single element) - input args for the function,<br>
d is derivative (if known),<br>
args are additional arguments for function (if required).<br>
Example:
<source lang=python>
from FuncDesigner import *
from scipy import fft
from numpy import arange
a, b, c = oovars('a', 'b', 'c')
FFT  = oofun(lambda x, y, z: (fft((x+2*y+4*z)*arange(16))).real, input=[a, b, c])
f = a**2+b**2+c**2 + FFT
point = {a:1, b:2, c:3} 
print f(point)
print f.D(point)

#  Another way, via creating oofun constructor:

from FuncDesigner import *
from scipy import fft
from numpy import arange, hstack
a, b, c = oovars('a', 'b', 'c')
myFFT = lambda INPUT: oofun(lambda *args:fft(hstack(args)).real, input=INPUT)
f = a**2+b**2+c**2 + myFFT([a+2*b, a+3*c, 2*b+4*c])
point = {a:1, b:2, c:3} 
print f(point)
print f.D(point)
</source>


= Interpolation oofun made from scipy.interpolate.UnivariateSpline =
This is example of using scipy.interpolate.UnivariateSpline. This function has been wrapped by a routine from FuncDesigner and yielded the oofun FuncDesigner.interpolate.scipy_UnivariateSpline with exactly same args/kwargs. See [http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.UnivariateSpline.html scipy.interpolate.UnivariateSpline doc] for whole list of possible parameters. You can easily get derivatives via [[AD|automatic differentiation]] and involve the function into numerical optimization problems.
<source lang = python>
from FuncDesigner import *
a, b, c = oovars('a', 'b', 'c') 
point1 = {a:1, b: 0, c:[1, 2, 3]}

mySpline = interpolate.scipy_UnivariateSpline([1, 2, 3, 4], [1.001, 4, 9, 16.01])

f = mySpline(a)

print(f(point1))
print(f.D(point1))

f2 = a + sin(b) + c[0] + arctan(1.5*f)
F = mySpline(a + f2 + 2*b + (c**2).sum())

print(F(point1))
print(F.D(point1))

"""
Expected output:
 [ 1.001]
{a: array([ 2.0015])}
[ 329.61795391]
{a: array([ 108.51234474]), b: array([ 111.39024934]), c: array([ 111.39024934,  148.52033245,  222.78049868])}
"""
</source>


= Boolean oofuns =
{{attention}}Some features mentioned here are [[coming]], some work in other way with FuncDesigner0.17/OpenOpt0.27 releases (so updating from svn repository or those zip-files from installation webpage are required).<br>
Boolean oofuns are created while using operations '>', '>=', .eq (equals), '<', '<=' on ordinary smooth oofuns. Maybe in Python 3.x series where Unicode is available I'll add something better than '.eq', currently I don't want to use '=' and '==' because of some issues.<br>
Let's note: since FuncDesigner is intended primarily for smooth functions, currently operations '>' and '>=' are same, as well as '<' and '<=' - all they yield "true" for equality case.<br>
Resulting oofun has output of type boolean (True/False); for vectorized comparisons it is False if '''any''' of vector coordinates equals to numpy.nan ("not a number", e.g. result of extracting logarithm or square root from negative number, result of user-defined function) or the constraint is violated:
<source lang=python>
from FuncDesigner import *
a, b = oovars(2)
point = {a:[2,4],b:[5,10]}
c = a+2*b; print(c(point)) # prints [12. 24.]
d = c > 15; print(d(point)) # prints '''False''' because c has coordinate 12 that is less than 15
z = c > -10; print z(point) # prints '''True''' because all c coordinates are greater than -10 in the point involved
z2 = log(b-7); print(z2(point)) # prints '''False''' because log([-2,3]) is [NaN,  1.09861229]
</source>





= Numerical optimization: using OpenOpt =
Currently you can use only the classes mentioned below, in nearest future I intend to provide connections and examples for some other OO classes (mb [[QP]], [[DFP]], [[NLLSP]], [[MINLP]]).  

Let me remember you - required constraints tolerance can be change during prob assignment, e.g. p = NLP(..., contol = 1e-7). See also the doc entry [[FuncDesignerDoc#Personal_tolerances_for_constraints|Personal tolerances for constraints]].<br>
Using FuncDesigner you don't have to provide 1st derivatives - they are calculated via [[FuncDesignerDoc#Automatic_differentiation|Automatic differentiation]]. <br>
If possible, constraints are rendered into box-bound and linear ones. <br>
Suppose you have optimization variables a, b, c = oovars('a', 'b', 'c'). To extract optimal point coordinates anything below works (since FuncDesigner v. 0.27):
<source lang = python>
a_opt,  b_opt, c_opt = r(a, b, c)
a_opt,  b_opt, c_opt = r(a), r(b), r(c)
r('a'), r('b'), r('c')
r('a', 'b', 'c')
r[a], r[b], r[c]
r['a'], r['b'], r['c']
a(r), b(r), c(r)
</source>
{{FD_OPT_EXAMPLES}}


== Personal tolerances for constraints ==
([[coming]])<br>
Each boolean oofun created by comparison (less, greater, equal) of smooth oofuns has field ''tol'' (tolerance, threshold for triggering False), 0 by default (if the oofun is encountered in constraints passed to openopt, it will use value p.contol for these ones). If you change it to other value, FuncDesigner and OpenOpt will use that one. You can assign it directly (c.tol = myValue), but other ways are more convenient and recommended:
<source lang=python>
from FuncDesigner import *
a = oovar()
point = {a: 1.5001}
# you can use any of the ways below
c = a<1.5; c.tol = 1e-3 # direct way, inconvenient - you have to type 'c' 2 times, and sometimes name is much longer.
c = (a<1.5)(tol=0.001) # to change tolerance you can call it with keyword arg '''tol'''
# in the __call__ routine you can assign constraint name as well:
c = (a<1.5)('asdf', tol=0.001) #   (*)
c = (a<1.5)('asdf')(tol=0.001) #   (**)
c = (a<1.5)(name='my constraint', tol=0.001)
c = (a<1.5)(tol=0.001, name='asdf')
c = (a<1.5)(name='asdf')(tol=1e-3)
# Most recommended ways are (*) and (**)
d = (a<1.5)(tol=0.000001)
print('c: ' + str(c1(point)) + '   d: ' + str(d(point))) # c: True   d: False
print((a+2).eq(3.50005)(tol=1e-10)(point)) # False
</source>
Providing adequate constraints tolerances is highly important for most of numerical optimization solvers. Lots of them will have difficulties with constraints of huge typical values (like 10<sup>3</sup>, 10<sup>4</sup> etc) while required tolerances for them are small (like 10<sup>-6</sup>, that is default for OpenOpt); or you have some of your constraints to be satisfied very exactly (like 10<sup>-20</sup>) while others are huge as above. Thus, when you assign your own tolerances, OpenOpt will scale it automatically to the working tolerance p.contol (via multiplying it with a known constant number).<br>
If you have strict constraint, that cannot be violated for any tol (e.g. x>=0 for to keep x<sup>0.25</sup> inside definition domain) but you still want to provide a non-default scaling factor for the constraint, you can use negative tol, e.g.
<source lang=python>
from FuncDesigner import *
from openopt import NLP
a = oovar()
objective = log(a-10000000000) + 100*exp(1/(a-10000000000)) # numbers around 10^10 dollars are involved
c = sqrt(a) > 1e5 + 0.1
c2 = (a>10000000000)(tol = -1000) # tolerance is 1000 dollars, strict inequation to keep objective defined
startPoint = {a:10000000001} 
p = NLP(objective, startPoint, constraints = [c, c2], maxIter = 1e4, ftol = 1e-4)
r = p.minimize('ipopt')
print(r.ff)
# My computer with IPOPT 3.8.0 prints 109.90849 for tol = -1000
# that is better than 109.9183 for tol = 1e-6 (default)
</source>
In future some FD functions will have automatically accompanied domain constraints.<br>
{{attention}} Known issues: currently assigning non-zero tol forces box-bound constraints (lb <= x <= ub) to be rendered as general linear (Ax<=b); to be fixed in future.



= Some issues to be aware =
* You can use f.size for oovars/oofuns but not len(f)
* For current FuncDesigner release (0.17) you can use the following indexation for oofuns/oovars: a[ind], a[ind1:ind2]. ind1 and ind 2 should be integers, positive or negative; you can't use a[ind:] or a[:ind] for now, and currently ind '''cannot''' be oovar/oofun, like a.size. Latter is intended to be fixed in next release.
* some_oovar[some_ind] < some_val is rendered into general linear constraint, not box-bound; the one is intended to be solved in future FuncDesigner/OpenOpt releases, alone with some other features/enhancements to be done.
* FuncDesigner creates lots of auxilary functions and class objects. Since CPython interpreter is slow in comparison with C/C++/Fortran etc, if you have problems with calculations speed you could be interested in [http://forum.openopt.org/viewtopic.php?id=67 Unladen Swallow] project, they say it already works with [[NumPy]]/[[SciPy]] and promise even more speed in release 2009Q3.
* For FuncDesigner 0.15 sum([oovar_0,...oovar_k]) works good but sum([oofun_0,...oofun_k]) works slow (requires O(n^2) operations, n is length of the array) and, moreover, can yield some problems because of stack overflow ("maximum recursion depth exceeded" or "oofun instance has no attribute 'f_key_prev'"). It works much better in release 0.17.
* Some enhancements for fixed oovars/oofuns and for recalculating oofuns in same points sequentially are intended to be done in next FD release (some ones are already done in 0.17).
* Currently general linear functions rendered into constraints A x <= b, Aeq x = beq use dense numpy arrays, even with scipy installed. This will be reworked in future releases, and possibility to get oofun.D() result of a scipy.sparse type will be added.
Templates used on this page:
Return to FuncDesignerDoc.
Retrieved from "http://openopt.org/FuncDesignerDoc"
View source

From OpenOpt

Views

Personal tools

Latest OOSuite 0.27

Development

Misc

Free Python-written numerical optimization software suite. Visitors since 2007/10/24:
