Markov控制过程在紧致行动集上的迭代优化算法-控制与决策.PDFVIP

第18 卷第3 期 　控　制　与　决　策 2003 年5 月 Vol. 18 No.3 　Control and D ecis ion　 　May 2003 : 1001-0920(2003) 03-0267-05 Markov 唐　昊, 奚宏生, 殷保群 ( , 230026) : 研究 一类连续时间Markov 控制过程(CTM CP) 在紧致行动集上关于平均代价 能准则的优 化算法。根据CTM CP 的 能势公式和平均代价最优 方程, 导出了求解最优或次最优平稳控制策略的 策略迭代算法和数值迭代算法, 在无需假设迭代算子是sp-压缩的条件下, 给出了这两种算法的收敛 证明。最后通过分析 一个受控排队网络的例子说明了这种方法的优越 。 : Markov 控制过程; 紧致行动集; 能势; 策略迭代; 数值迭代 : TP202: Iteration optimization algorithms for Markov control processes with compact action set TA N G H ao, X I H ong -she ng , YIN Bao-qun ( Department of utomation U niversity of Science and Technology of China Hefei, , 230026,China) : - Abstract Optimization algorithms are studied for a class of continuous time Markov control processes (CTM CPs) with infinite horizon average-cost criteria and compact action set. By using the formula of performance potentials and an average-cost optimality equation for CTMCPs, a policy iteration algorithm and a value iteration algorithm are derived which can lead to an optimal or suboptimal, stationary policy in a finite number of iterations T he convergence of these algorithms is established. , without the assumption of the corresponding iteration operator being an sp-contraction. numerical example of queuing networks show s advantages of the proposed value iteration method. Key words Markov control processes Compact action set Performance potentials Policy iteration: ; ; ; ; Value iteration 1 引　　言 Markov , Markov (MCP) MCP , [5, 6] MCP(DT MCP) MCP(CTMCP) MCP ,