PLAN.md - Telescope Auction System Implementation¶

Project Overview¶

Build a microservices-based telescope auction system where telescopes bid on observation tasks using vector-based capability matching. No ML - all configuration-based with manual ranges from astronomy literature.

Phase 1: Core Vector System (Week 1)¶

Goal: Get vector conversion and similarity working standalone

Day 1-2: Vector Configuration & Utilities¶

[ ] Create project structure

  telescope-auction/
  ├── vector_system/
  │   ├── __init__.py
  │   ├── vector_config.py
  │   ├── vector_utils.py
  │   ├── telescope_vectorizer.py
  │   ├── task_vectorizer.py
  │   ├── similarity.py
  │   └── task_categories.py
  ├── tests/
  │   ├── test_vectorization.py
  │   └── test_similarity.py
  ├── docs/
  │   └── literature_references.md
  └── requirements.txt

[ ] File: vector_config.py
- Define DimensionConfig dataclass
- Define VectorConfig class with all 7 dimensions
- Add literature references for each range
- Add sub-metric ranges (Bortle, seeing, etc.)
- Test: Print all configs, verify ranges make sense
[ ] File: vector_utils.py
- Implement normalize_value() with clamping
- Implement normalize_aperture()
- Implement normalize_wavelength_range()
- Implement normalize_resolution() (inverse)
- Implement normalize_location_quality() (composite)
- Implement normalize_instruments()
- Implement normalize_mount_precision()
- Implement normalize_fov()
- Test: Each function with edge cases (below min, above max, exactly at boundaries)
[ ] File: similarity.py
- Implement cosine_similarity()
- Implement weighted_cosine_similarity()
- Implement euclidean_distance() (optional, for comparison)
- Implement meets_minimum_threshold()
- Implement check_hard_requirements()
- Test: Known vectors with known expected results

Day 3: Telescope Vectorizer¶

[ ] File: telescope_vectorizer.py
- Implement TelescopeVectorizer class
- Implement vectorize() method
- Implement vectorize_batch() for multiple telescopes
- Implement vector_to_dict() for debugging
- Implement vectorize_with_warnings() for outlier detection
- Test: Create 5 sample telescopes with different specs, verify vectors

Day 4: Task Vectorizer & Categories¶

[ ] File: task_categories.py
- Define TASK_CATEGORY_WEIGHTS dict with all 6 categories:
  - deep_sky_imaging
  - planetary
  - exoplanet_transit
  - spectroscopy
  - wide_field_survey
  - time_domain
- Add descriptions, weights, rationale, and references for each
- Implement get_weights_for_category()
- Implement get_category_info()
- Test: Verify all categories return 7 weights
[ ] File: task_vectorizer.py
- Implement TaskVectorizer class
- Implement vectorize() method
- Implement vectorize_with_category() (returns vector + weights)
- Implement vector_to_dict() for debugging
- Test: Create tasks for each category, verify different weight sets

Day 5: Integration Testing & Documentation¶

[ ] File: test_vectorization.py
- Test complete telescope → vector → comparison pipeline
- Test with 3 telescopes and 3 tasks from different categories
- Verify scores make intuitive sense
- Test outlier handling (FOV > max, aperture < min)
[ ] File: docs/literature_references.md
- Document all astronomy references used
- Add URLs to online sources
- Add notes on why each range was chosen
[ ] Create example script: example_matching.py
- Show complete workflow from specs to match scores
- Include explain_match() function for transparency

Phase 2: Database Layer (Week 2)¶

Goal: Persist telescopes, tasks, and vectors

Day 6-7: Database Schema & Models¶

[ ] Choose database: PostgreSQL (supports array types for vectors)
[ ] File: database/schema.sql

sql

  CREATE TABLE telescopes (
      id UUID PRIMARY KEY,
      name VARCHAR(255) NOT NULL,
      owner_id UUID,

      -- Raw specifications (for display)
      aperture_mm FLOAT NOT NULL,
      wavelength_min_nm FLOAT,
      wavelength_max_nm FLOAT,
      resolution_arcsec FLOAT,
      bortle_scale INTEGER,
      seeing_arcsec FLOAT,
      clear_nights_per_year INTEGER,
      instruments TEXT[],
      mount_accuracy FLOAT,
      fov_degrees FLOAT,

      -- Normalized vector (for matching)
      capability_vector FLOAT[7] NOT NULL,

      -- Metadata
      status VARCHAR(20) DEFAULT 'available', -- available, busy, maintenance
      current_queue UUID[], -- task IDs in queue
      has_outlier_values BOOLEAN DEFAULT false,
      outlier_dimensions TEXT[],

      created_at TIMESTAMP DEFAULT NOW(),
      updated_at TIMESTAMP DEFAULT NOW()
  );

  CREATE TABLE tasks (
      id UUID PRIMARY KEY,
      submitter_id UUID,
      category VARCHAR(50) NOT NULL,
      priority_level VARCHAR(20) DEFAULT 'normal', -- normal, high

      -- Raw requirements
      min_aperture_mm FLOAT,
      wavelength_min_nm FLOAT,
      wavelength_max_nm FLOAT,
      max_resolution_arcsec FLOAT,
      min_location_quality FLOAT,
      required_instruments TEXT[],
      min_tracking_precision FLOAT,
      min_fov_degrees FLOAT,

      -- Observation details
      target_name VARCHAR(255),
      target_coordinates POINT, -- RA, Dec
      time_window_start TIMESTAMP,
      time_window_end TIMESTAMP,
      exposure_time_seconds INTEGER,

      -- Normalized vector & weights
      requirement_vector FLOAT[7] NOT NULL,
      dimension_weights FLOAT[7] NOT NULL,
      min_threshold FLOAT DEFAULT 0.7,

      -- Auction settings
      auction_duration_seconds INTEGER DEFAULT 300,
      auction_started_at TIMESTAMP,
      auction_status VARCHAR(20) DEFAULT 'pending', -- pending, active, closed, assigned

      -- Assignment
      assigned_telescope_id UUID REFERENCES telescopes(id),
      assigned_at TIMESTAMP,
      completed_at TIMESTAMP,

      created_at TIMESTAMP DEFAULT NOW()
  );

  CREATE TABLE bids (
      id UUID PRIMARY KEY,
      task_id UUID REFERENCES tasks(id) ON DELETE CASCADE,
      telescope_id UUID REFERENCES telescopes(id),

      telescope_vector FLOAT[7],
      bid_score FLOAT NOT NULL,

      status VARCHAR(20) DEFAULT 'active', -- active, withdrawn, accepted, rejected

      submitted_at TIMESTAMP DEFAULT NOW(),
      withdrawn_at TIMESTAMP,

      UNIQUE(task_id, telescope_id) -- One bid per telescope per task
  );

  CREATE TABLE auction_events (
      id UUID PRIMARY KEY,
      task_id UUID REFERENCES tasks(id),
      event_type VARCHAR(50), -- auction_started, bid_received, auction_closed, winner_selected
      event_data JSONB,
      timestamp TIMESTAMP DEFAULT NOW()
  );

  -- Indexes
  CREATE INDEX idx_telescopes_status ON telescopes(status);
  CREATE INDEX idx_tasks_status ON tasks(auction_status);
  CREATE INDEX idx_tasks_priority ON tasks(priority_level);
  CREATE INDEX idx_bids_task ON bids(task_id);
  CREATE INDEX idx_bids_score ON bids(bid_score DESC);

[ ] File: database/models.py (using SQLAlchemy)
- Define Telescope model
- Define Task model
- Define Bid model
- Define AuctionEvent model
- Add helper methods (e.g., telescope.to_dict())
[ ] Test: Create, read, update, delete operations for each model

Day 8: Repository Layer¶

[ ] File: database/repositories/telescope_repository.py
- create_telescope(raw_specs, capability_vector)
- get_telescope(telescope_id)
- get_available_telescopes()
- update_telescope_status(telescope_id, status)
- update_telescope_queue(telescope_id, queue)
[ ] File: database/repositories/task_repository.py
- create_task(raw_requirements, requirement_vector, weights, category)
- get_task(task_id)
- get_tasks_by_status(status)
- update_task_status(task_id, status)
- assign_task(task_id, telescope_id)
[ ] File: database/repositories/bid_repository.py
- create_bid(task_id, telescope_id, score, telescope_vector)
- get_bids_for_task(task_id)
- get_winning_bid(task_id) (highest score)
- withdraw_bid(bid_id)
- withdraw_all_bids_for_telescope(telescope_id) (when telescope wins elsewhere)

Phase 3: Core Microservices (Week 3)¶

Goal: Build individual services with REST APIs

Day 9-10: Telescope Registry Service¶

[ ] Tech stack: Python + FastAPI + PostgreSQL
[ ] File: services/telescope_registry/main.py
- Setup FastAPI app
- Add database connection
- Add health check endpoint
[ ] Endpoints:
- POST /telescopes - Register new telescope
  - Accepts raw specs
  - Vectorizes automatically
  - Stores both raw + vector
  - Returns telescope_id and vector
- GET /telescopes/{id} - Get telescope details
  - Returns raw specs (for humans)
  - Optionally returns vector (for services)
- GET /telescopes - List all telescopes
  - Filter by status (available, busy, maintenance)
- PATCH /telescopes/{id}/status - Update availability
  - available → busy (when wins bid)
  - busy → available (when completes task)
- GET /telescopes/{id}/queue - Get task queue
- PATCH /telescopes/{id}/queue - Update queue
[ ] Test: Use Postman/curl to test all endpoints

Day 11-12: Task Registry Service¶

[ ] File: services/task_registry/main.py
[ ] Endpoints:
- POST /tasks - Submit new observation task
  - Accepts raw requirements + category
  - Vectorizes requirements
  - Gets weights for category
  - Stores task
  - Returns task_id
- GET /tasks/{id} - Get task details
- GET /tasks - List tasks
  - Filter by status, priority
- PATCH /tasks/{id}/status - Update task status
- POST /tasks/{id}/assign - Assign task to telescope
  - Update task status to 'assigned'
  - Add task to telescope queue
[ ] Test: Create tasks in different categories, verify vectors and weights

Day 13-14: Bidding Service¶

[ ] File: services/bidding/main.py
[ ] Endpoints:
- POST /bids/evaluate - Check if telescope can bid
  - Input: telescope_id, task_id
  - Fetch telescope vector from DB
  - Fetch task vector and weights from DB
  - Calculate similarity score
  - Check against threshold
  - Return: {can_bid: bool, score: float, reason: string}
- POST /bids - Submit bid
  - Input: telescope_id, task_id
  - Evaluate first (can_bid check)
  - If yes, create bid record
  - Return bid_id and score
- DELETE /bids/{id} - Withdraw bid
  - Mark bid as withdrawn
- GET /bids/task/{task_id} - Get all bids for task
  - Sorted by score descending
[ ] Business Logic:
- calculate_bid_score(telescope_vector, task_vector, task_weights)
- meets_threshold(score, threshold)
- meets_hard_requirements(telescope_vec, task_vec) for critical dimensions
[ ] Test: Create multiple telescopes bidding on same task, verify scoring

Phase 4: Auction Engine (Week 4)¶

Goal: Orchestrate the bidding process

Day 15-16: Auction Engine Service¶

[ ] File: services/auction_engine/main.py
[ ] Endpoints:
- POST /auctions - Create auction for a task
  - Input: task_id
  - Set task status to 'active'
  - Broadcast to eligible telescopes (via event bus)
  - Start timer for auction_duration
  - Return auction_id
- GET /auctions/{task_id}/status - Get auction status
  - Current bid count
  - Time remaining
  - Leading bid
- POST /auctions/{task_id}/close - Close auction
  - Calculate winner (highest bid score)
  - Assign task to winning telescope
  - Withdraw all other bids from this task
  - Update telescope status to 'busy'
  - Remove telescope from all other active auctions
  - Emit auction.closed event
[ ] Background Workers:
- Auction timer worker
  - Monitors active auctions
  - Auto-closes when auction_duration expires
- Broadcast worker
  - When auction starts, check all available telescopes
  - For each, call bidding service to evaluate
  - If can_bid, notify telescope (via websocket or poll)
[ ] Test:
- Create task with 5min auction
- Have 3 telescopes bid
- Verify highest score wins
- Verify losers are removed from other auctions

Day 17: Queue Management Service¶

[ ] File: services/queue_manager/main.py
[ ] Endpoints:
- GET /telescopes/{id}/queue - Get current queue
- POST /telescopes/{id}/queue/add - Add task to queue
  - When telescope wins a bid
  - Sort by priority (high → normal)
- POST /telescopes/{id}/queue/remove - Remove task
  - When outbid elsewhere
- POST /telescopes/{id}/queue/reorder - Manual priority adjustment
[ ] Logic:
- When telescope wins Task A:
  1. Add Task A to queue
  2. Call auction engine to withdraw all other active bids for this telescope
  3. If queue has multiple tasks, sort by priority
- When telescope completes a task:
  1. Remove task from queue
  2. If queue empty, mark telescope as 'available'
  3. If queue has tasks, start next task
[ ] Test: Telescope wins 2 tasks, verify queue ordering

Phase 5: Supporting Infrastructure (Week 5)¶

Goal: Add message queue, API gateway, notifications

Day 18-19: Message Queue & Events¶

[ ] Setup RabbitMQ or Kafka
[ ] Define Events:
- auction.started - {task_id, auction_duration, requirements}
- bid.submitted - {bid_id, task_id, telescope_id, score}
- auction.closed - {task_id, winner_telescope_id, winning_score}
- task.assigned - {task_id, telescope_id}
- task.completed - {task_id, telescope_id}
- telescope.available - {telescope_id}
[ ] Publishers:
- Auction engine publishes auction.started, auction.closed
- Bidding service publishes bid.submitted
- Task registry publishes task.assigned, task.completed
[ ] Subscribers:
- Bidding service listens to auction.started → auto-evaluates for available telescopes
- Queue manager listens to auction.closed → updates queues
- Notification service listens to all events → sends updates

Day 20: API Gateway¶

[ ] Tech: Nginx or Kong or FastAPI aggregate
[ ] Routes:
- /api/telescopes/* → Telescope Registry Service
- /api/tasks/* → Task Registry Service
- /api/bids/* → Bidding Service
- /api/auctions/* → Auction Engine
- /api/queue/* → Queue Manager
[ ] Add:
- Rate limiting
- Authentication/authorization (JWT)
- Request logging
- CORS config

Day 21: Notification Service¶

[ ] File: services/notifications/main.py
[ ] Capabilities:
- WebSocket connections for real-time updates
- Email notifications (optional)
- Webhook support (optional)
[ ] WebSocket Events:
- Connect telescope clients
- When auction.started → send to all available telescopes
- When bid.accepted → send to winning telescope
- When bid.rejected → send to losing telescopes
- When task.assigned → send to both submitter and telescope

Phase 6: Frontend & Testing (Week 6)¶

Goal: Build basic UI and end-to-end tests

Day 22-23: Admin Dashboard¶

[ ] Tech: React or simple HTML + JavaScript
[ ] Pages:
- Telescope list (view all telescopes, their status, queues)
- Task list (view all tasks, their status, bids)
- Create task form (select category, input requirements)
- Auction monitor (live view of active auctions)
- Match explainer (show dimension-by-dimension breakdown)

Day 24: Telescope Client Simulator¶

[ ] File: clients/telescope_simulator.py
- Simulate multiple telescopes
- Auto-bid on compatible tasks
- Accept task assignments
- Report task completion
[ ] Use for testing:
- Spawn 10 simulated telescopes
- Create 20 tasks
- Watch auction process
- Verify correct matching

Day 25-26: End-to-End Testing¶

[ ] Test Scenarios:
1. Basic auction flow
  - Register 3 telescopes
  - Submit 1 deep-sky task
  - Verify all 3 bid
  - Verify highest score wins
2. Priority handling
  - Submit normal priority task
  - Submit high priority task while normal is active
  - Verify high priority closes faster
3. Queue management
  - Telescope wins 2 tasks
  - Verify both in queue, priority order
  - Telescope wins 3rd task elsewhere → verify removed from first 2
4. Outlier handling
  - Register telescope with FOV = 50° (outside range)
  - Verify it gets normalized score of 1.0
  - Verify it can still match tasks
5. Category weight differences
  - Same telescope bids on planetary vs. deep-sky
  - Verify different scores based on weights
6. No valid bids
  - Submit task requiring 5000mm aperture
  - Only 2000mm telescopes available
  - Verify auction stays