Lecture12:Planning

1.PDDLplanninglanguage p g g g
2.Progressionandregressionplanning
3.Partialorderplanning
4 SAT Planning 4.SATPlanning
ITUniversityofCopenhagen
Evaluation Evaluation Evaluation Evaluation
ITUniversityofCopenhagen
2011RuneMllerJensen 2
TodaysProgram TodaysProgram TodaysProgram TodaysProgram
[12:0013:00]RN1010.1,10.2
PDDL PlanningLanguage
Planning as statespace search Planningasstate space search
Progressionplanning(forwardsearch)
Regression planning (backward search) Regressionplanning(backwardsearch)
[13:1014:00]RN0311.3,RN1010.410.5
Pl i l h Planningasplanspacesearch
Partialorderplanning
SATPlanning
ITUniversityofCopenhagen
2011RuneMllerJensen 3
Planning Planning Planning Planning
Problem
Howtosequenceactionstoachievegivengoalstates
In principle just a search problem but Inprinciplejustasearchproblem,but
Statesarefactored
ACTION andRESULT functionsaredefinedinaction
schemas
Actionschemasrevealproblemstructure
Actionsoftenrepresentrealactivities
Search trees often have huge branching factors
ITUniversityofCopenhagen
Searchtreesoftenhavehugebranchingfactors
4 2011RuneMllerJensen
Applications Applications Applications Applications
Containershipredeployment
Sortieoperations
Failurerecovery
ITUniversityofCopenhagen
5 2011RuneMllerJensen
AICoreMethods AICoreMethods AICoreMethods AICoreMethods
HowcoveredinIAIP
Perception Perception
-DecisionTreeLearning
Knowledge
PropositionalLogic
Bi d i i di
Perception
K l d
Binarydecisiondiagrams
Rules
configurationrules
constraintsolverrules
Knowledge
Reasoning
Planningdomains
Reasoning
-HeuristicSearch
SATsolving
Tasks
g
Constraintsolving
Binarydecisiondiagram
construction
configuration
Tasks
Tasks
g
PlanningAlgorithms
Gameplaying
ITUniversityofCopenhagen
-Plans
6 2011RuneMllerJensen
PlanningLanguages PlanningLanguages PlanningLanguages PlanningLanguages
Provide declarative definitionsof
1. Planningdomains
A f i h h i h Asetofactions,whereeachactionhas
aprecondition definingwhenitcanbeapplied
aneffect defininghowtheactionchangesthestate
2 Planning problems 2. Planningproblems
Initial stateandgoalstatesofaplanning
domains
ITUniversityofCopenhagen
2011RuneMllerJensen 7
PlanningSystems PlanningSystems PlanningSystems PlanningSystems
Input
Output
p
p
Planning
Planning
Domain
Planning
System
Domain
Planning
Plan
Planning
Problem
ITUniversityofCopenhagen
2011RuneMllerJensen 8
Planning PlanningLanguageObjectives LanguageObjectives Planning PlanningLanguageObjectives LanguageObjectives
Whatisagoodlanguage? g g g
1. Expressive enoughtodescribeawidevarietyof
bl problems
2. Restrictive enough to allow efficient algorithms 2. Restrictive enoughtoallowefficientalgorithms
tooperateonit
3. Planningalgorithmshouldbeabletotake
advantageofthelogicalstructure ofthe g g
problem
ITUniversityofCopenhagen
9 2011RuneMllerJensen
PDDLPlanningLanguage PDDLPlanningLanguage PDDLPlanningLanguage PDDLPlanningLanguage
ITUniversityofCopenhagen
10 2011RuneMllerJensen
Predicatesand PredicatesandFluents Fluents Predicatesand PredicatesandFluents Fluents
Apredicate isaparameterized propositional symbol
Examples
i i ( ) ( ) i I i i () university(x)at(x,y)itIsRaining()
equal(x,y)plus(x,y,z)
AFluent isagroundpredicate(eithertrueorfalse)
university(ITU)at(ITU,Amager)
itIsRaining()equal(4,4) plus(3,4,7) g() q ( , ) p ( , , )
ITUniversityofCopenhagen
2011RuneMllerJensen 11
PDDLPlanningLanguageAssumptions PDDLPlanningLanguageAssumptions PDDLPlanningLanguageAssumptions PDDLPlanningLanguageAssumptions
Closedworldassumption:
Astateisaconjunctionoftruefluents,allother
fluents are assumed to be false fluentsareassumedtobefalse
Actionsaredeterministic
Environmentsarestatic
S b bl Statesareobservable
States and actions are discrete Statesandactionsarediscrete
Actionsareinstantaneous
ITUniversityofCopenhagen
2011RuneMllerJensen 12
RepresentationsofActions RepresentationsofActions RepresentationsofActions RepresentationsofActions
Action=Precondition+Effect
Precondition
C j ti f lit l (t d f l fl t ) th t Conjunctionofliterals(trueandfalsefluents)that
mustbetrueinthecurrentstatefortheactiontobe
applicable applicable
Effect
Conjunctionofliteralsthatdefineswhichfluents are
addedanddeletedfromthecurrentstatewhen
applyingtheaction
ITUniversityofCopenhagen
13 2011RuneMllerJensen
RepresentationsofActions(cont.) RepresentationsofActions(cont.) RepresentationsofActions(cont.) RepresentationsofActions(cont.)
ActionSchema=parameterizedaction p
Example
ActionFly(p,s,d)
PRECOND: At(p s) . Plane(p) . PRECOND: At(p,s). Plane(p).
Airport(s). Airport(d)
EFFECT: At(p s) . At(p d) EFFECT: At(p,s). At(p,d)
We will always assume actions to be grounded
when planning: RN03 and RN10 does not!
ITUniversityofCopenhagen
14 2011RuneMllerJensen
p g
RepresentationofInitialStateandGoal RepresentationofInitialStateandGoal RepresentationofInitialStateandGoal RepresentationofInitialStateandGoal
Aninitialstate isa singlestate
Ex:At(Truck1,Melbourne). In(Box1,Truck1).
Box(Box1). Connected(Sydney,Melbourne)
Agoal isa conjunctionofliterals thatmustbe
i l trueinagoalstate
Ex:At(Box1,Sydney) . At(Truck1,Sydney)
We will always assume goals to be grounded We will always assume goals to be grounded.
RN03 and RN10 does not!
ITUniversityofCopenhagen
15 2011RuneMllerJensen
Example: Example:Air AirCCargo argoTTransport ransport Example: Example:Air AirCCargo argoTTransport ransport
P1
P2
SFO
JFK
C1
P2
C2
Init = At(C1 SFO) . At(C2 JFK) . At(P1 SFO) . At(P2 JFK) . Cargo(C1) . Cargo(C2) . Plane(P1) . Init =At(C1,SFO). At(C2,JFK). At(P1,SFO). At(P2,JFK). Cargo(C1). Cargo(C2). Plane(P1).
Plane(P2). Airport(JFK). Airport(SFO)
Goal =At(C1,JFK). At(C2,SFO)
Action:loadcargofromairporttoairplane
Action: unload cargo from airplane to airport Action:unloadcargofromairplanetoairport
Action:flyairplanebetweenairports
ITUniversityofCopenhagen
16 2011RuneMllerJensen
Example: Example:AirCargoTransport AirCargoTransport Example: Example:AirCargoTransport AirCargoTransport
Init =At(C1,SFO). At(C2,JFK). At(P1,SFO). At(P2,JFK). Cargo(C1). Cargo(C2). Plane(P1)
. Plane(P2). Airport(JFK). Airport(SFO)
Goal=At(C1,JFK). At(C2,SFO)
ActionLoad(c,p,a)
PRECOND:Cargo(c). Plane(p). Airport(a). At(c,a). At(p,a)
EFFECT:At(c,a). In(c,p)
Action Unload(c,p,a)
PRECOND:Cargo(c). Plane(p). Airport(a). In(c,p). At(p,a)
EFFECT:At(c,a). In(c,p)
Action Fly(p,s,d)
PRECOND:Plane(p). Airport(s). Airport(d). At(p,s)
EFFECT: At(p,s). At(p,d)
ITUniversityofCopenhagen
17 2011RuneMllerJensen
ActionApplicability ActionApplicability ActionApplicability ActionApplicability
Anactiona isapplicable inanystates thatsatisfy pp y y
itsprecondition:
a e ACTIONS(s) s PRECOND(a) a e ACTIONS(s) s PRECOND(a)
Example: p
state:Plane(P1). Plane(P2). Airport(JFK). Airport(SFO) .
At(P1,JFK). At(P2,SFO)
Satisfiespreconditionof :Fly(P1,JFK,SFO) = Plane(P1).
Airport(JFK). Airport(SFO). At(P1,JFK)
ThusFly(P1,JFK,SFO) isapplicableinthestate!
ITUniversityofCopenhagen
18 2011RuneMllerJensen
ActionExecution ActionExecution ActionExecution ActionExecution
Theresult ofexecutingactiona instates is:
\ RESULT(s,a)=(s \ DEL(a))ADD(a) where
ADD(a) andDEL(a) arethepositiveandnegative
fluents intheeffect ofa,respectively
Example Example
s=Plane(P1). Plane(P2). Airport(JFK)
. Airport(SFO). At(P1,JFK). At(P2,SFO)
a =Action Fly(P1,JFK,SFO)
PRECOND:Plane(P1). Airport(JFK). Airport(SFO). At(P1,JFK)
EFFECT: At(P1,JFK) . At(P1,SFO) EFFECT: At(P1,JFK). At(P1,SFO)
RESULT(s,a)=Plane(P1). Plane(P2). Airport(JFK)
. Airport(SFO) . At(P1,SFO). At(P2,SFO)
ITUniversityofCopenhagen
p ( ) ( , ) ( , )
19 2011RuneMllerJensen
Example: Example:AirCargoTransport AirCargoTransport Example: Example:AirCargoTransport AirCargoTransport
Init =At(C1,SFO). At(C2,JFK). At(P1,SFO). At(P2,JFK). Cargo(C1). Cargo(C2). Plane(P1).
Plane(P2). Airport(JFK). Airport(SFO)
Goal=At(C1,JFK). At(C2,SFO)
ActionLoad(c,p,a)
P1
SFO
PRECOND:At(c,a). At(p,a). Cargo(c). Plane(p). Airport(a)
EFFECT:At(c,a). In(c,p)
A ti U l d( )
C1
Action Unload(c,p,a)
PRECOND:In(c,p). At(p,a). Cargo(c). Plane(p). Airport(a)
EFFECT: At(c,a) . In(c,p) EFFECT:At(c,a). In(c,p)
Action Fly(p,s,d)
PRECOND:At(p,s). Plane(p). Airport(s). Airport(d)
P2
JFK
EFFECT: At(p,s). At(p,d)
Plan = Load(C1,P1,SFO),Fly(P1,SFO,JFK), Unload(C1,P1,JFK)
C2
ITUniversityofCopenhagen
Load(C2,P2,JFK), Fly(P2,JFK,SFO), Unload(C1,P1,JFK)
20 2011RuneMllerJensen
Planningas PlanningasState StateSpace SpaceSSearch earch Planningas PlanningasState StateSpace SpaceSSearch earch
ITUniversityofCopenhagen
21 2011RuneMllerJensen
Progressionand ProgressionandRegressionPlanning RegressionPlanning Progressionand ProgressionandRegressionPlanning RegressionPlanning
Search node = plan state
Forward
Search node = plan state
At(P1 JFK) . At(P1 SFO) .
Forward
(Progression)
At(P1,JFK) .
Plane(P1) . Airport(SFO)
. Airport(JFK)

b
At(P1,SFO) .
Plane(P1) . Airport(SFO)
. Airport(JFK)
Fly(P1,SFO,JFK)
Backward
At(P1,SFO) .
Plane(P1) . Airport(SFO) b At(P1,JFK)
Fly(P1,SFO,JFK)
(Regression)
. Airport(JFK)

ITUniversityofCopenhagen
22 2011RuneMllerJensen
Search node = sub goals
Progression ProgressionPlanning Planning Progression ProgressionPlanning Planning
Potentially Inefficient (thought so 1961 1998) PotentiallyInefficient(thoughtso19611998)
Irrelevantactionsmaybeapplied
Oftenhugebranchingfactor
H i ti ( h l 1994 ) Heuristics(graphplan 1994>)
Heuristicsfromrelaxedproblems p
Ignoredeletelist
Ignore precondition Ignoreprecondition
ITUniversityofCopenhagen
23 2011RuneMllerJensen
Regression RegressionAlgorithm(STRIPS) Algorithm(STRIPS) Regression RegressionAlgorithm(STRIPS) Algorithm(STRIPS)
Mainadvantage:onlyconsiderrelevantactions
muchlowerbranchingfactor!
Apply action backward if Applyactionbackwardif
1. Relevant : GoalAdd=
2 Consistent Goal Del 2.Consistent:GoalDel =
Resulting new goal (stop search if Goal _ Init) Resultingnewgoal(stopsearchifGoal _ Init)
Goal'=(Goal\ Add)Pre
Recall: We assume actions are grounded!
STRIPS: only positive literals in preconditions and goal
ITUniversityofCopenhagen
24 2011RuneMllerJensen
These slides on regression is syllabus, not RN10 10.2.2
Regression RegressionAlgorithm(STRIPS) Algorithm(STRIPS) Regression RegressionAlgorithm(STRIPS) Algorithm(STRIPS)
Example p
Goal= At(C1,B). At(C2,B). . At(C20,B)
Relevant action for first predicate: Relevant actionforfirstpredicate:
Unload(C1,P1,B)
PRE: Cargo(C1) . Plane(P1) . Airport(B) . In(C1 P1) . At(P1 B) PRE:Cargo(C1). Plane(P1). Airport(B). In(C1,P1). At(P1,B)
EFF:At(C1,B). In(C1,P1)
Goal'= Cargo(C1). Plane(P1). Airport(B). In(C1,P1).
At(P1,B). At(C2,B). . At(C20,B)
load(C1,P1) isinconsistent sinceitdeletesAt(C1,B)
ITUniversityofCopenhagen
25 2011RuneMllerJensen
Heuristicsfor HeuristicsforState StateSpace SpaceSSearch earch Heuristicsfor HeuristicsforState StateSpace SpaceSSearch earch
HeuristicsforSTRIPSRegressionPlanning
Ignoredellistsandperformaforwardsearch
Cost of predicate P equals shortest path to P in the CostofpredicateP equalsshortestpathtoP inthe
relaxedforwardsearch(easytofind!)
The he ristic al e of a s b goal in a regression Theheuristicvalueofasubgoalinaregression
searchisthesum/maxofthecostofthe
di t i th l predicatesinthegoal
h
add
:fast
add
h
max
:admissible
ITUniversityofCopenhagen
26 2011RuneMllerJensen
Heuristicsfor HeuristicsforState StateSpace SpaceSSearch earch Heuristicsfor HeuristicsforState StateSpace SpaceSSearch earch
ADD andMAX HeuristicsforRegressionPlanning
b
a
c
a=2, c=1, p=0
q p
a
c
G
c
p
, , p
h
add
= 2 + 1 + 0
3
Init: q, p
= 3
h
max
= 2
Act1: p c
Act2: c q b a
max
Act2: c,q b,a

ITUniversityofCopenhagen
27 2011RuneMllerJensen
Planning as Planning as Plan Plan Space Search Space Search Planning as Planning as Plan Plan Space Search Space Search Planningas PlanningasPlan PlanSpaceSearch SpaceSearch
(RN0311.3 (RN0311.3 notes) notes)
Planningas PlanningasPlan PlanSpaceSearch SpaceSearch
(RN0311.3 (RN0311.3 notes) notes) (( )) (( ))
ITUniversityofCopenhagen
28 2011RuneMllerJensen
Partial PartialOrder OrderPPlanning lanning Partial PartialOrder OrderPPlanning lanning
Progressionandregressionplanners
Sequenceactionsinatotalorder,eventhough
ti b d d bit il someactionsmaybeorderedarbitrarily
PartialOrderPlanners(POP) ( )
Orderactionspartiallysuchthatactionscanbe
d i i f i h d sequencedinanywaysatisfyingtheorder
Least commitment strategy Leastcommitmentstrategy
Onlyaddstrictlynecessaryorderingconstraints
ITUniversityofCopenhagen
29 2011RuneMllerJensen
Shoe ShoeExample Example Shoe ShoeExample Example
Goal=RightShoeOn . LeftShoeOn
i Init=
Action RightSock
PRECOND:
Action LeftSock
PRECOND:
EFFECT:RightSockOn
Action RightShoe
EFFECT:LeftSockOn
Action LeftShoe
PRECOND L f S kO
PRECOND:RightSockOn
EFFECT:RightShoeOn
PRECOND:LeftSockOn
EFFECT:LeftShoeOn
Planner:combinetwoactionsequences
(1)leftSock,leftShoe
(2)rightSock,rightShoe
ITUniversityofCopenhagen
30 2011RuneMllerJensen
ShoeExample ShoeExample ShoeExample ShoeExample
ITUniversityofCopenhagen
31 2011RuneMllerJensen
POP POPasa asaSearch SearchPProblem roblem POP POPasa asaSearch SearchPProblem roblem
Searchnodesarepartialplans
Th t l t i l Theemptyplancontainsonlya
startaction:Pre={},Eff=init
finishaction:Pre=goal,Eff={}
Eachsearchnodehas4components: p
Asetofactions (stepsoftheplan)
A set of ordering constraints: A < B Asetoforderingconstraints:A <B
Asetofcausallinks:
A set of open preconditions
A
p
B
Asetofopenpreconditions
ITUniversityofCopenhagen
32 2011RuneMllerJensen
PartialPlanSearchNodeExample PartialPlanSearchNodeExample PartialPlanSearchNodeExample PartialPlanSearchNodeExample
rightSockOn rightShoeOn
S
RightShoe
rightShoeOn
Start
Finish
LeftSock LeftShoe
leftSockOn
leftSockOn leftShoeOn
leftShoeOn
Actions: {RightShoe, LeftSock, LeftShoe, Start, Finish}
Orderings: {LeftSock < LeftShoe, Start < X, X < Finish}
Causal link: {LeftSock-leftSockon->LeftShoe,
RightShoe-rightShoeOn->Finish, LeftShoe-leftShoeOn->Finish}
Open preconditions {rightSockOn}
ITUniversityofCopenhagen
Open preconditions {rightSockOn}
33 2011RuneMllerJensen
Solving SolvingPOP POP Solving SolvingPOP POP
Initialsearchnode:containsStart andFinish,the
orderingconstraintStart <Finish,nocausallinks,all
thepreconditionsinFinish areopen
Successorfunction:
picks one open precondition p of an action B picksoneopenpreconditionp ofanactionB
generatesasuccessorplanstateforeverypossible
i f h i i A h hi consistent wayofchoosingactionA thatachievesp
(obs: A maybetakenfromtheplan!)
Goaltest:noopenpreconditionsleft
ITUniversityofCopenhagen
34 2011RuneMllerJensen
Enforcing EnforcingConsistency Consistency Enforcing EnforcingConsistency Consistency
Whengeneratingsuccessorplanstate:
1) Addthecausallink andtheordering
constraint A < B to the plan.
A
p
B
constraintA B totheplan.
IfA isnew,alsoaddstart <A,A <Finish
2) Resolve conflicts between new causal link and all 2) Resolveconflictsbetweennewcausallinkandall
existingactions
reorder actions that threaten it (has p as effect) reorderactionsthatthreatenit(hasp aseffect)
3) IfA isnew:resolveconflictsbetweenactionA and
ll i ti l li k allexistingcausallinks
reorderA ifitthreatenssomeofthem
ITUniversityofCopenhagen
35 2011RuneMllerJensen
Example:Spare Example:SpareTire TirePProblem roblem Example:Spare Example:SpareTire TirePProblem roblem
Init=At(Flat,Axle). At(Spare,trunk)
Goal = At(Spare Axle) Goal=At(Spare,Axle)
Action Remove(Spare,Trunk)
Action PutOn(Spare,Axle)
PRECOND:At(Spare,Trunk)
EFFECT:At(Spare,Trunk) . At(Spare,Ground)
PRECOND:At(Spare,Ground). At(Flat,Axle)
EFFECT:At(Spare,Axle). At(Spare,Ground)
Action Remove(Flat,Axle)
PRECOND A (Fl A l )
Action LeaveOvernight
PRECOND:At(Flat,Axle)
EFFECT:At(Flat,Axle). At(Flat,Ground)
PRECOND:
EFFECT: At(Spare,Ground). At(Spare,Axle)
. At(Spare,trunk). At(Flat,Ground).
( l l )
ITUniversityofCopenhagen
36 2011RuneMllerJensen
At(Flat,Axle)
SAT SATPlanning Planning SAT SATPlanning Planning
ITUniversityofCopenhagen
37 2011RuneMllerJensen
Idea Idea Idea Idea
1. DefineaBooleanvariableforeachgroundpredicate andinstantiatedaction
2. timestampthevariableswithstamp0to i
3. Addconstraintsofactions
4 Add i f l d i i i l 4. Addconstraintsofgoalandinitialstate
5. Checkifthecorrespondingformulaissatisfiable,ifso,aplanexistoflengthi
0 1
i

at(P2,JFK)
0
at(P1,SFO)
0
Fly(P1,SFO,JFK)
0
Fly(P1,JFK,SFO)
0
Fly(P2,JFK,SFO)
0
at(P2,JFK)
1
at(P1,SFO)
1
Fly(P1,SFO,JFK)
1
Fly(P1,JFK,SFO)
1
Fly(P2,JFK,SFO)
1

at(P2,JFK)
i
...
at(P1,SFO)
i
Init atoms must
be true
Goal atoms must
be true

...

...
ITUniversityofCopenhagen
38 2011RuneMllerJensen
SATPLANalgorithm SATPLANalgorithm SATPLANalgorithm SATPLANalgorithm
function SATPLAN(problem,T
max
) return solution or failure
input:problem,aplanningproblem
T
max
, an upper limit to the plan length T
max
,anupperlimittotheplanlength
for T=0 to T
max
do
CN i ANSLA O SA ( bl ) CNF,mapping TRANSLATETOSAT(problem,T)
assignment SATSOLVER(CNF)
if assignment isnotnullthen
return EXTRACTSOLUTION(assignment, mapping) return EXTRACT SOLUTION(assignment,mapping)
return failure
ITUniversityofCopenhagen
39 2011RuneMllerJensen
HowtoencodeactionsintoPL HowtoencodeactionsintoPL HowtoencodeactionsintoPL HowtoencodeactionsintoPL
1. Specifyinitialstatefully
At(P1,SFO)
0
. At(P2,JFK)
0
. At(P1,JFK)
0
. At(P2,SFO)
0
2. Define the goal states as the goal expression 2. Definethegoalstatesasthegoalexpression
At(P1,JFK). At(P2,SFO)
dd 3. Addasuccessorstateaxioms
F
t+1
(AaddF
1
t
v v AaddF
n
t
)v (F
t
. (AdelF
1
t
v v AdelF
m
t
))
4. Foreachaction,addpreconditionaxioms
Fly(P2,SFO,JFK)
0
At(P2,SFO)
0
5. At mostoneactionappliedineachtimestep
(Fly(P1 SFO JFK)
0
. Fly(P2 SFO JFK)
0
) . (Fly(P1 JFK SFO)
0
. Fly(P2 JFK SFO)
0
) .
ITUniversityofCopenhagen
(Fly(P1,SFO,JFK) . Fly(P2,SFO,JFK) ). (Fly(P1,JFK,SFO) . Fly(P2,JFK,SFO) ).
40 2011RuneMllerJensen