i know there has been a lot of abstract theory developed, usually with proportional transaction costs. do people just sort of manipulate the frictionless models in an ad hoc way or are there any systematic models used?

When designing a strategy is it essential to include realistic transaction costs. However for pricing and equilibrium models transaction costs are generally not used, except at the end to explain discrepancies.

In practice, transaction cost is often included in reserve calculation (which is deducted from P&L from day 1 and dynamically thereafter). Usually what reserves try to capture is the bid/offer for the underlying product being measured, i.e. if you wanted to unload the product now, what would the loss be (under "normal" market circumstances). When this is the case, none of the hedging parameters reflect potential transaction cost, but the adjusted PV does in it's own way, provided there are some close enough products that are liquid) In credit products, the reserves are generally unhedged and increase as spreads widen, decrease as they tighten (as bid offer in absolute terms goes down, in addition to other risks). This is one of the many reasons the investment banks have made a killing on the run in the market this year.