Team 01

1. Description & Scope

Our project, "Fusion Robotics," establishes an autonomous testbed integrating a quadruped robot and a UR5 robotic arm, orchestrated using ROS 2. The core objective is to create a repeatable, automated workflow for testing quadruped locomotion, collecting sensor data, and evaluating performance through a closed loop involving real-world execution and analysis.

1.1 Project Scoping and Evolution

Initially, our project broadly aimed to explore sensor fusion for optimizing experimental setups. Through the initial development cycles (Weeks 7-12), involving individual component testing (UR5 control, quadruped basic motion, sensor readings), we identified the critical need to first solidify the core autonomous workflow. Therefore, our scope refined to prioritize the seamless integration and interaction between the UR5 and the quadruped for the automated reset sequence. This became the primary milestone, ensuring a stable platform before delving deeper into complex simulation comparisons or advanced SLAM implementations initially envisioned. The focus shifted towards robust hardware integration, real-time control via a user interface, and reliable data acquisition through ROS 2.

1.2 Data Collection Process

Our data collection process is central to the experimental loop:

1. Initialization: The UR5 arm precisely places the quadruped robot at a designated starting position.
2. Autonomous Locomotion: Triggered via our ROS 2 system (potentially through the UI), the quadruped begins walking forward based on pre-set or user-defined gait parameters (See Section 4).
3. Real-time Sensing: As the quadruped moves, its onboard sensors (IMU, Camera - providing data for /orb/pose, Servo encoders) continuously publish data to respective ROS 2 topics.
4. Data Logging & Visualization: A dedicated ROS 2 node (/ros2_flask_plotter) subscribes to these topics, logging the data and visualizing key metrics (IMU readings, pose estimates) in real-time via a web interface.
5. Path Completion & Reset: Upon reaching a predefined distance or condition (monitored via pose estimation), the quadruped stops. A signal is sent (potentially automated or via UI) to the UR5 arm.
6. Automated Reset: The UR5 executes a pre-programmed routine to pick up the quadruped and return it to the starting position.
7. Iteration: The cycle repeats, allowing for continuous data collection under varying parameters or conditions.

Initial data collection involved testing sensors individually and calibrating the quadruped's basic movements. This data was crucial for tuning low-level controllers and identifying necessary filtering techniques. We found raw IMU data too noisy due to motor vibrations, necessitating filtering. Early camera pose tests highlighted sensitivity to lighting, guiding our focus towards robust estimation methods. This iterative process of collect-analyze-refine was essential and required revisiting initial data assumptions.

1.3 Pose Estimation and Model Fitting

Accurate pose estimation is vital for path completion detection and performance analysis. We employ multiple approaches:

• IMU-based Pose Estimation: The /pose_estimator_node integrates data from /imu/data and /imu/orientation (likely using techniques like Kalman filtering or complementary filters) to produce /imu/pose_estimate. This provides a continuous estimate of the robot's orientation and potentially position relative to its start.
• Camera-based Pose Estimation: The /optical_flow_pose_publisher node processes camera data (likely visual features or optical flow) to publish pose information on the /orb/pose topic. This offers an alternative, potentially drift-correcting pose source, especially useful for tracking position over ground.
• Sensor Fusion (Implicit/Comparative): While the current ROS graph doesn't show explicit fusion of /imu/pose_estimate and /orb/pose into a single topic, our web interface plots both. This allows for direct comparison and validation. Future work could involve fusing these using filters (e.g., Extended Kalman Filter) for a more robust estimate. Data collected informs the tuning of these estimators (e.g., filter noise parameters).

1.4 Integration into ROS 2

The entire system is orchestrated using ROS 2 Humble. The quadruped itself is integrated via the /bridge_node, which translates high-level commands and parameter changes into low-level /servo/command messages and publishes /servo/state. Sensor data is published directly by dedicated nodes or drivers (e.g., IMU node, camera node publishing to /orb/pose). A Flask web server (/ros2_flask_plotter) acts as the central UI, subscribing to relevant topics for visualization and potentially publishing commands or parameter updates. (See Section 4 for the detailed architecture).

1.5 Validation and Success Metrics

We quantify success through several metrics:

• Workflow Completion: Successfully executing multiple consecutive cycles of the walk-and-reset loop autonomously.
• Pose Estimation Accuracy: Comparing /imu/pose_estimate and /orb/pose. While ground truth from OptiTrack (as mentioned in original goals) might not be fully integrated yet, we assess consistency, drift characteristics, and agreement between the methods. Graphical plots showing the time evolution of both estimates are used for visual validation.
• Control Responsiveness: Demonstrating real-time changes in quadruped gait parameters (Frequency, Duty Factor, Amplitudes, Offsets, Mode) via the web UI and observing the corresponding immediate change in locomotion.
• Data Integrity: Ensuring sensor data logged via the /ros2_flask_plotter is consistent and reflects the robot's state accurately.

2. Updated Project Goals

Our core vision of creating an advanced, automated robotic testbed remains. However, based on initial development and feasibility assessments, our project goals have been refined since the beginning:

• Original Goal: Develop a testbed optimizing experimental accuracy via sensor fusion and simulation comparisons.
• Updated & Current Goals:
  1. Demonstrate a Fully Autonomous Workflow: Successfully implement and showcase the closed-loop system where the UR5 positions the quadruped, the quadruped walks autonomously while collecting data, and the UR5 resets it for continuous operation.
  2. Implement Real-Time Quadruped Control via UI: Provide a user-friendly web interface (Flask-based) that allows real-time monitoring of sensor data (IMU, Pose Estimates) and interactive control over the quadruped's gait parameters (Frequency, Duty Factor, Offsets, Amplitudes) and movement modes (Walk, Trot, Pronk, etc.).
  3. Integrate and Compare Pose Estimation Methods: Utilize and log pose estimates derived from the IMU (/imu/pose_estimate) and camera-based methods (/orb/pose), enabling comparison and analysis of their performance within our specific setup.
  4. Develop Robust ROS 2 Integration: Create a modular and reliable ROS 2 architecture connecting the UR5 (control assumed via existing ROS drivers/custom scripts), the quadruped (via /bridge_node), sensors, estimation nodes, and the user interface.
  5. Provide Open Resources: Share our developed ROS 2 packages, setup instructions, and collected experimental data to benefit the robotics community (as originally planned).

The emphasis has shifted from deep simulation integration within this semester to proving the core hardware automation, real-time control, and multi-sensor data handling pipeline first.

3. Project Process / Workflow

Our project workflow follows an iterative development process within the ROS 2 ecosystem. The high-level plan remains similar, but the focus within the later weeks shifted towards integration and UI development.

Note: While the overall weekly themes remain similar to the initial plan, Weeks 12-15 saw an increased focus on developing the bridge_node for parameter control, the ros2_flask_plotter UI, and achieving the core UR5-Quadruped automated reset loop, deferring deeper simulation comparisons.

Project Timeline (Weeks 7-16)

4. Finalized System & ROS 2 Architecture

Our system leverages ROS 2 Humble for modular communication between the hardware components, control logic, and user interface. The architecture is designed for clarity and extensibility.

Key Components:

• Nodes (Ellipses/Rectangles in Diagram):
  • /bridge_node (Dark Grey): Interfaces directly with the quadruped's hardware (servos). Subscribes to /servo/command and parameter updates. Publishes /servo/state. Handles gait generation based on parameters.
  • /pose_estimator_node (Purple): Subscribes to /imu/data, /imu/orientation. Performs calculations (e.g., sensor fusion, integration) and publishes /imu/pose_estimate.
  • /optical_flow_pose_publisher (Teal): Processes camera data (implied input) using optical flow/visual features. Publishes pose to /orb/pose.
  • /ros2_flask_plotter (Pink): Bridge between ROS 2 and the web UI. Subscribes to sensor/state topics (/imu/*, /orb/pose, /servo/state) for visualization. Hosts services/publishers for UI commands and parameter changes (possibly via /parameter_events).
  • (Implied Nodes): Sensor driver nodes publishing raw data (IMU, Camera), UR5 control node.
• Topics (Arrows connecting Nodes, grouped by namespaces):
  • /imu/data (Yellow): Raw IMU accelerometer/gyroscope data.
  • /imu/orientation (Yellow): Filtered IMU orientation (e.g., quaternions).
  G
  • /imu/pose_estimate (Yellow): Calculated pose from IMU data (by /pose_estimator_node).
  • /orb/pose (Teal): Calculated pose from camera/visual data (by /optical_flow_pose_publisher).
  • /servo/command (Red): Commands sent to quadruped servos (by /bridge_node).
  • /servo/state (Red): Current state from servos (published by /bridge_node).
  • /parameter_events (Grey): Standard ROS 2 topic for parameter changes.
  • /rosout (Grey): Standard ROS 2 logging topic.

Quadruped Control Parameters (via UI):

System Architecture Visualization

Interact with the diagram or view the static architecture.

Initializing...

IMU Pose

Camera Pose

UR5e Arm

❮ ❯

Data Flow Example (Control):

1. User changes 'Frequency' parameter on the Web UI.
2. UI sends command via Flask backend to /ros2_flask_plotter.
3. /ros2_flask_plotter updates the relevant parameter on /bridge_node (e.g., via parameter service call or /parameter_events).
4. /bridge_node receives the parameter update, adjusts its internal gait generator logic.
5. /bridge_node publishes new joint targets on /servo/command.
6. Quadruped hardware responds to new commands.
7. Changes in motion affect /imu/* and /orb/pose data, which are visualized back on the UI via /ros2_flask_plotter.

5. System Tradeoffs & Technical Considerations

Engineering this integrated robotic system involved balancing key technical requirements:

1. Pose Estimation: Accuracy vs. Resources
   • Conflict: High-accuracy SLAM demands significant computation (CPU/RAM), conflicting with real-time needs on potentially limited onboard hardware. Basic IMU integration is fast but drifts significantly.
   • Balance: Implemented parallel, independent IMU-based (fast filter, drift-prone position) and camera-based (/orb/pose, likely VO, less drift, higher latency/load) estimators. Deferred complex real-time fusion; enabled comparison via UI.
   • Implication: Provides redundancy and comparison data but relies on simpler/less optimal estimates for real-time control logic or requires offline analysis for best accuracy.
2. Real-time Control vs. Stability
   • Conflict: Instantaneous UI changes to gait parameters (frequency, amplitude) can command dynamically unstable states for the quadruped.
   • Balance: Used ROS 2 parameters managed via the UI. The /bridge_node likely applies rate limiting, value clamping, and/or input smoothing to parameter changes before generating servo commands.
   • Implication: Prioritizes robot stability and safety over immediate command response, resulting in smoother, controlled transitions.
3. Hardware Constraints vs. Algorithm Choice
   • Conflict: Onboard computing limits (e.g., Raspberry Pi) restrict the feasibility of running complex algorithms like full SLAM or MPC concurrently with control loops.
   • Balance: Focused on core functionality using performant, established ROS 2 packages and less demanding algorithms (e.g., filters/VO instead of full SLAM). Deferred resource-intensive features.
   • Implication: Achieved reliable core operation on target hardware, but performance ceilings (accuracy, speed) may be lower than systems with high-end compute.
4. Development Time vs. Feature Scope
   • Conflict: Limited time required prioritizing core features over implementing the full initial vision (e.g., deep simulation, advanced fusion).
   • Balance: Focused on delivering the Minimum Viable System: the robust automated walk-reset loop, multi-sensor data pipeline, interactive UI control, and comparative pose estimation.
   • Implication: Delivered a functional, integrated testbed demonstrating the core concept, providing a validated platform for future, more advanced feature integration.
5. Path End Detection: Robustness vs. Precision (Type I & Type II Error)
   • Conflict: Stopping the quadruped accurately using noisy/drifting pose estimates risks stopping too early (Type I error) or overshooting the UR5 pickup zone (Type II error).
   • Balance: Prioritized successful automated resets. Employed conservative stopping logic (e.g., filtering/hysteresis on pose estimate, slightly shorter target distance) and a generous UR5 pickup envelope.
   • Implication: High reliability in the autonomous loop, minimizing collision/failure risk, at the cost of potentially slightly shorter travel distances per run.