ManualTrader/docs/trade-lifecycle.md

Trade Lifecycle Management

This document explains how the trading bot manages the lifecycle of a trade, from its initiation to its conclusion, including various states, updates, and handling of special conditions like stop losses and external exchange activities.

Core Components

The trade lifecycle is primarily managed by the interplay of three core components:

1. src/trading/trading_engine.py: Responsible for initiating trade actions (placing entry, exit, stop-loss orders) and interacting with the exchange client. It relies on TradingStats for state persistence.
2. src/trading/trading_stats.py: The brain for state management. It uses an SQLite database (data/trading_stats.sqlite) to store and update the status of orders and trade lifecycles. Key tables are orders and trades.
3. src/monitoring/market_monitor.py: Continuously monitors the exchange for events like order fills, cancellations, and external activities. It updates the state in TradingStats based on real-world exchange events.

Database Schema for Lifecycle Tracking

Two main tables in trading_stats.sqlite are central to lifecycle management:

1. orders Table

Stores information about individual orders placed by the bot or detected. Key fields:

• id (INTEGER, PK): Database ID for the order.
• bot_order_ref_id (TEXT, UNIQUE): A UUID generated by the bot for internal tracking before an exchange ID is available.
• exchange_order_id (TEXT, UNIQUE): The order ID provided by the exchange.
• symbol (TEXT): Trading pair (e.g., "ETH/USDC:USDC").
• side (TEXT): 'buy' or 'sell'.
• type (TEXT): Order type (e.g., 'market', 'limit', 'stopMarket', 'stopLimit', 'stop_loss_trigger').
• status (TEXT): Critical for lifecycle. Examples:
  • pending_submission: Bot intends to submit, not yet confirmed by exchange.
  • open: Actively working on the exchange.
  • filled: Order fully executed.
  • partially_filled: Order partially executed.
  • cancelled: Order cancelled (by bot, user, or system).
  • rejected: Exchange rejected the order.
  • expired: Order expired.
  • pending_trigger: A conditional order (like a stop loss) whose trigger conditions have not yet been met.
  • failed_submission: Bot failed to submit the order to the exchange.
• parent_bot_order_ref_id (TEXT, NULLABLE): Links conditional orders (like SL triggers) to their parent entry order. This helps maintain context, especially for stop losses attached to a specific trade.
• timestamp_created (TEXT), timestamp_updated (TEXT).

2. trades Table

Stores information about entire trade lifecycles, which can consist of one or more orders (e.g., entry, TP, SL). Key fields:

• id (INTEGER, PK): Database ID for the trade record.
• trade_lifecycle_id (TEXT, UNIQUE): A UUID grouping all related activities (entry, exit, adjustments) for a single conceptual trade.
• symbol (TEXT).
• status (TEXT): Critical for lifecycle. Examples:
  • pending: Lifecycle initiated, entry order may be pending_submission or open.
  • executed: Entry order has been placed (usually for market orders that are expected to fill quickly).
  • position_opened: Entry order confirmed filled, position is active on the exchange.
  • position_closed: Position has been fully closed (via TP, SL, or manual exit).
  • cancelled: The trade was cancelled before a position was opened, or an open position's management was explicitly cancelled.
• position_side (TEXT): 'long', 'short'. Becomes 'flat' implicitly when position_closed.
• entry_price (REAL), current_position_size (REAL).
• stop_loss_price (REAL), take_profit_price (REAL).
• entry_order_id (TEXT), stop_loss_order_id (TEXT), take_profit_order_id (TEXT): Exchange order IDs for the respective parts of the trade.
• realized_pnl (REAL), unrealized_pnl (REAL).
• position_opened_at (TEXT), position_closed_at (TEXT).
• linked_order_table_id (INTEGER): Foreign key to the orders table, linking a specific fill event to this trade record.

Trade Lifecycle Stages & Status Transitions

Here's a typical flow of a trade and how its status evolves:

1. Initiation (e.g., User sends /long ETH 100 sl 1800)

• TradingEngine:
  • Generates a unique bot_order_ref_id for the entry order.
  • Calls TradingStats.record_order_placed():
    • An entry is made in the orders table.
    • orders.status = pending_submission.
    • orders.bot_order_ref_id is set.
    • If a stop loss is part of the initial command (stop_loss_price provided):
      • A separate order may be recorded in orders with type='stop_loss_trigger', status='pending_trigger', and parent_bot_order_ref_id linking it to the entry order's bot_order_ref_id.
      • Alternatively, the stop loss might be implicitly tracked against the trade lifecycle.
  • Calls TradingStats.create_trade_lifecycle():
    • An entry is made in the trades table.
    • trades.trade_lifecycle_id is generated.
    • trades.status = pending (or executed if it's a market order).
    • trades.symbol, trades.position_side are set.
    • trades.stop_loss_price is recorded if provided.

2. Order Submission to Exchange

• TradingEngine:
  • Attempts to place the entry order via hyperliquid_client.
  • On Success:
    • Receives exchange_order_id from the exchange.
    • Calls TradingStats.update_order_status():
      • orders.status updated to open (for limit orders) or a transient status like submitted_market (market orders await fill confirmation from MarketMonitor).
      • orders.exchange_order_id is populated.
    • The trades table entry (created in step 1) remains pending or executed.
  • On Failure (e.g., insufficient funds, API error):
    • Calls TradingStats.update_order_status():
      • orders.status updated to failed_submission or failed_submission_no_data.
    • Calls TradingStats.update_trade_cancelled():
      • trades.status updated to cancelled.
      • The lifecycle ends here.

3. Exchange Acknowledges Order (Order is Live)

• MarketMonitor:
  • In its periodic _update_cached_data() and _check_order_fills() loop, it sees the new order (with exchange_order_id) as open on the exchange.
  • The orders.status is typically open.

4. Entry Order Fill Detection

• MarketMonitor (_check_order_fills, _process_filled_orders):
  • Detects that the entry exchange_order_id is no longer open (implying it was filled or cancelled).
  • It verifies against recent fills from the exchange.
  • Assuming it was filled:
    • Calls TradingStats.update_order_status():
      • orders.status (for the entry order) updated to filled.
      • orders.amount_filled is updated.
    • Calls TradingStats.update_trade_position_opened():
      • trades.status updated to position_opened.
      • trades.entry_price, trades.current_position_size, trades.position_opened_at are recorded.
      • trades.entry_order_id is set to the exchange_order_id of the filled entry order.
      • If a stop loss was pre-defined (e.g., stop_loss_price in the trades table), and a pending_trigger SL order exists in the orders table, it remains in that state, now actively monitored against the open position.

5. Position is Open and Managed

• The trade is now live (trades.status = 'position_opened').
• MarketMonitor continues its checks:
  • Pending Stop Loss Monitoring (_activate_pending_stop_losses_from_trades, _check_pending_triggers):
    • If a stop_loss_price is defined in the trades table for this position_opened lifecycle (or a linked pending_trigger order exists in the orders table).
    • MarketMonitor checks the current market price against this stop_loss_price.
    • If Triggered:
      • MarketMonitor instructs TradingEngine.execute_triggered_stop_order() (or a similar method).
      • TradingEngine places a new market or limit stop order on the exchange.
      • This new SL order is recorded in the orders table by TradingEngine (via TradingStats.record_order_placed()), likely with status='open' and type='stopMarket' or 'stopLimit'.
      • TradingStats.link_stop_loss_to_trade() is called to update trades.stop_loss_order_id with the exchange_order_id of this newly placed active SL order.
      • The original pending_trigger order (if one existed) in the orders table is updated to triggered or cancelled_by_activation.

6. Position Closure (SL, TP, or Manual Exit)

• Scenario A: Stop Loss Order Fills
  • MarketMonitor detects the SL exchange_order_id (from trades.stop_loss_order_id) has filled.
  • Calls TradingStats.update_order_status() for the SL order: orders.status = filled.
  • Calls TradingStats.update_trade_position_closed():
    • trades.status = position_closed.
    • trades.realized_pnl, trades.position_closed_at, and exit price details are recorded.
  • If a Take Profit order was also active for this lifecycle, TradingStats.cancel_linked_orders() is called to cancel it. The TP order's status in orders becomes cancelled_linked_fill (or similar).
• Scenario B: Take Profit Order Fills
  • Similar to SL fill, but for the TP order. The linked SL order is cancelled.
• Scenario C: Manual Exit Command (e.g., /exit ETH)
  • TradingCommands calls TradingEngine.execute_exit_order().
  • TradingEngine places a market order to close the position. This new exit order goes through the pending_submission -> open (if limit) / submitted_market -> filled flow in the orders table.
  • When MarketMonitor detects this exit order fill:
    • Calls TradingStats.update_trade_position_closed(): trades.status = position_closed.
    • TradingStats.cancel_pending_stop_losses_by_symbol() or cancel_linked_orders() is called to cancel any active SL/TP orders associated with this trade_lifecycle_id. Their status in orders becomes cancelled_position_closed (or similar).

7. Order Cancellation

Orders can be cancelled at various stages:

• By Bot User (e.g., /coo command for "Cancel All Orders" for a symbol):
  • TradingCommands calls TradingEngine.cancel_all_orders().
  • TradingEngine calls client.cancel_order() for each open order.
  • TradingStats.update_order_status(): orders.status becomes cancelled.
  • If an entry order is cancelled before opening a position, TradingStats.update_trade_cancelled() sets trades.status to cancelled.
• By System (e.g., Orphaned Stop Loss):
  • MarketMonitor._cleanup_orphaned_stop_losses(): If a position (trades.status = 'position_closed' or 'cancelled') has a lingering SL order that is still open or pending_trigger.
  • MarketMonitor instructs TradingEngine to cancel it on the exchange.
  • TradingStats.update_order_status() for the SL order: orders.status = cancelled_orphaned.
• Detected by MarketMonitor (e.g., user cancelled on exchange interface):
  • MarketMonitor._process_disappeared_orders(): If an order known to the bot vanishes from the exchange without a fill.
  • After checks, if determined to be a cancellation:
    • TradingStats.update_order_status(): orders.status = cancelled_external (or a similar status indicating external action).
    • If this was a critical entry order, trades.status might also become cancelled.

Handling External Trades/Positions (Outside Telegram/Bot Control)

The system is designed to detect and adapt to trading activity that occurs directly on the exchange, not initiated by the bot.

• External Fill Detection (MarketMonitor._check_external_trades):
  • MarketMonitor fetches recent exchange fills.
  • It compares these fills with orders known to the bot (via bot_order_ref_id or exchange_order_id in the orders table).
  • If a fill does not match any bot order:
    • It's treated as an external trade.
    • TradingStats.record_trade() is used to log this activity. This might:
      • Create a new entry in the trades table if it opens a new position. The status would become position_opened, and entry details are estimated from the fill.
      • Update an existing trades entry if the external fill closes or modifies a bot-known position. For example, if the bot has trades.status = 'position_opened' for ETH, and an external sell fill for ETH is detected, TradingStats.update_trade_position_closed() would be called.
• Orphaned Position Sync (MarketMonitor._auto_sync_orphaned_positions):
  • Scenario 1: Exchange has position, Bot doesn't know.
    • MarketMonitor sees a position on the exchange that doesn't correspond to any position_opened lifecycle in trades table.
    • It calls TradingStats.create_trade_lifecycle() and TradingStats.update_trade_position_opened() to create a new record in trades, marking it as position_opened. Entry price is estimated from recent fills if possible. This ensures the bot is aware of the position for potential future management or correct P&L.
  • Scenario 2: Bot thinks position is open, Exchange doesn't.
    • MarketMonitor sees no corresponding position on the exchange for a trades entry that has status = 'position_opened'.
    • It calls TradingStats.update_trade_position_closed(), assuming the position was closed externally. Details like exit price might be approximate or marked as "external close."

Risk Management Integration

While the detailed logic for _check_automatic_risk_management in MarketMonitor isn't fully elaborated from the snippets, it would interact with this lifecycle:

• Modifying Orders: If risk rules dictate adjusting a stop loss or taking profit, MarketMonitor would instruct TradingEngine to cancel the existing SL/TP order and place a new one. This involves:
  • Cancelling the old order (updating its status in orders to cancelled_risk_adjusted or similar).
  • Placing a new order (new entry in orders table, status open).
  • Updating the relevant trades table fields (stop_loss_price, stop_loss_order_id, etc.).
• Preventing Trades: If a global risk limit (e.g., max exposure) is hit, TradingEngine might be prevented from initiating new trades, or MarketMonitor might flag this. The TradingEngine's order placement methods already include checks (e.g., for balance, valid prices) before attempting to submit orders.

Summary of Key Statuses

• orders.status: pending_submission, open, filled, cancelled (with variants like cancelled_external, cancelled_linked_fill, cancelled_orphaned, cancelled_position_closed), failed_submission, pending_trigger, triggered.
• trades.status: pending, executed, position_opened, position_closed, cancelled.

This detailed flow ensures that the bot maintains a robust internal state of its trading activities, reconciles with actual exchange events, and handles various scenarios gracefully.