Question

I wonder if the objective function of a general dynamic programming problem can always be formulated as in dynamic programming on wiki, where the objective function is a sum of items for action and state at every stage? Or that is just a specical case and what is the general formulation?

---

EDIT:

By “dynamic programming problem”, I mean a problem that can be solved by dynamic programming technique. Such kind of problems possess the property of optimal problem and optimal structure.

But at lease for me it is sometimes not easy to identify such problems, perhaps because I have not become used to that kind of verbal description. When I came across the WIKI page for Bellman equation, I do feel mathematical formulation of the cost function will help somehow. I suspect the overall cost/gain function can always be represented as accumulation of cost/gain from all the stages? and the accumulation can be additive or multiplitive or something else?

When I posted my question, I did realize that it is more proper to discuss dynamic programming in some place more oriented to mathematical optimization. But there are quite a lot of discussion of computer algorithms in Stackoverflow.com. So I did not feel improper to ask my question here either.

Answer 1

That’s not how I would characterize an arbitrary optimization problem (or a dynamic programming algorithm). In particular, the factor β^t looks like an electrical engineering hack that programmers wouldn’t usually want. More subtly, it seems like it won’t always be obvious what the function F is for a given problem.

But yes, set β to 1 and any arbitrary objective function can be formulated that way. Generally the objective function may be any function of the initial state and all the actions taken; given such a function, it’s easy to define a function F to plug into that formula.

Whether that’s a useful thing to do or not depends on the problem, I suppose.