Completely agree. You are modeling the real physical market, and how transactions is occurred. In a sense, it is like a Particle Swarm, when agents all agree on something, asset bubble occurs. I am based off the same thought processes, just eventually model the occurrence of a trade event. I can explain the density function of prices, but can not explain the volume. I actually tried your approach too. So I look at the volume / volatility relationship. It is promising.It's not a matter of what is or is not a model but whether a given model has a prayer of offering reasonably accurate insights into the physical system of interest.
Astronomers have no direct data on gravity but they can infer it's existence from its effects and build models of planetary motion that are much better than the epicyclic models they replaced. And market modelers actually do have a lot of data on agent actions in the form of the order book, the order flow, and various news media streams. Sure, the internal states of the agents may be hidden, but wouldn't you agree that a models of semi-stochastic agents submitting orders that then affect prices does a better job than assuming random exogenous price increments? For example, I've created agent models, too, and they have the nice property of recapitulating the coupling between price movement and trading volume as well as showing boom and bust cycles in cases in which agents "believe" in momentum.
Then I got stuck when deciding the # of agents -> which changes volume and vols. Also, each news can be interpret either good or bad. The direction of the agents is unknown too. But I think your approach is powerful that you can model volumes which in return reflect the manifested volatility. I can only look at "normalized behaviors" , i.e., fitted into vol ... or the total energy of a particle ball. Also bid/ask behaviors could be a great outcomes too.
I think all of us touch upon a key point is that market is not completely random, as you called semi-stochastic agents. And the interaction between randomness and deterministic path is not what has dictated financial market dx = u dt + s dW for the past couple of decades.