7. Career roles

After exploring the technology stack, consider how these skills come together at work. This chapter follows five groups: brain algorithms, motion control, perception and navigation, engineering and platforms, and hardware. Each role covers what it does, the technologies it uses, and how to prepare for interviews.

The interview topics below are preparation suggestions based on the skills each role requires. Check the job description for its research depth, engineering scope, and experience requirements.

7.1 “Brain” algorithm roles

Embodied foundation model / VLA engineer

Responsibilities: train and improve models that turn vision, language, and robot state into actions. Work includes architecture design, pretraining, fine-tuning, data mixtures, cross-embodiment training, and reinforcement learning post-training. Research roles emphasize methods and generalization; applied roles also connect models to hardware and evaluate them.

Common technologies: multimodal models, Transformers, imitation learning, reinforcement learning, PyTorch / JAX, and distributed training tools such as FSDP / DeepSpeed. Action representations include discrete tokens, diffusion, and flow matching; OpenVLA, RDT, and π0 illustrate different designs.

Interview preparation: understand one or two representative models in depth, including inputs, action spaces, training objectives, and inference. Be able to analyze memory, communication, and throughput in multi-GPU training, and explain data mixing, normalization, evaluation splits, and hardware failures. For RL post-training roles, also explain how rewards, value estimation, interaction data, and policy updates work together. Reproductions, training experiments, and hardware deployments can all provide evidence of these skills.

Related learning: Foundation model algorithms · π₀.₅ + RECAP project.

Manipulation engineer

Responsibilities: enable arms, bimanual systems, and dexterous hands to grasp, transport, assemble, and perform everyday tasks while handling object variation, occlusion, contact, and recovery. The work often combines geometric methods, motion planning, learned policies, and feedback control.

Common technologies: grasping work such as GraspNet / AnyGrasp, object 6D pose estimation, MoveIt 2, collision detection, ACT, Diffusion Policy, impedance and force control, and dexterous-hand motion retargeting.

Interview preparation: coordinate transformations, forward and inverse kinematics, trajectory planning, and contact control; how demonstrations affect training; and how action chunking works with closed-loop execution. Prepare a concrete task and explain whether failures came from perception, reachability, planning, policies, or control, including how you diagnosed and improved them.

Manipulation applies to industrial assembly, logistics, and service robots, with different precision, compliance, and generalization requirements. FieldAI's manipulation role illustrates responsibilities combining manipulation and whole-body coordination.

Related learning: Manipulation algorithms · ACT bimanual training.

7.2 “Cerebellum” algorithm roles

Motion control engineer

Responsibilities: make quadrupeds, bipeds, or humanoids balance, track velocity, cross different terrain, and coordinate whole-body motion. Look for two distinct emphases in job descriptions:

Direction	Main work	Common technologies
Learning-based / RL locomotion	Train motion policies, design rewards and curricula, and transfer policies from simulation to hardware	Isaac Lab, MuJoCo, PPO, domain randomization, motion imitation, and retargeting
Model-based control	Model robot dynamics and solve for control signals under contact and actuator constraints	MPC, whole-body control (WBC), numerical optimization, Pinocchio, OCS2, and CasADi

Interview preparation: for learning-based roles, understand PPO, advantage estimation, reward design, and training stability, and analyze sim-to-real problems caused by delays, noise, and actuator differences. For model-based roles, be able to derive kinematics, dynamics, and contact constraints, and explain optimization objectives and solver failures. Both require control-rate awareness, real-time execution, C++ hardware deployment, and log-based debugging.

Real systems can combine learned and model-based methods. FieldAI's locomotion role includes policy training, motion imitation, domain randomization, and hardware validation, illustrating responsibilities from training through deployment.

Related learning: Motion control algorithms · MicroDuck RL.

7.3 Perception and navigation roles

Navigation / SLAM engineer

Responsibilities: mapping, localization, state estimation, global and local path planning, obstacle avoidance, and recovery so robots can reliably reach goals.

Common technologies: lidar or visual-inertial methods such as FAST-LIO, LIO-SAM, ORB-SLAM3, and VINS; filtering and graph optimization; ROS 2, tf2, and Nav2. Vision-language navigation (VLN) and semantic navigation also involve language understanding, semantic maps, and appropriate evaluation.

Interview preparation: sensor calibration, coordinate frames, time synchronization, IMU errors, loop closure, and relocalization; the division between global and local planning; and diagnosing drift, localization degeneracy, dynamic obstacles, and narrow passages. Prepare log replays and failure analysis showing how problems were reproduced and fixed.

These skills overlap with navigation work in autonomous driving, AMRs, and household mobile robots, but sensors, speed ranges, and operating constraints differ. FieldAI's calibration, localization, and mapping role is an example of responsibilities in multisensor systems.

Related learning: Navigation.

Perception / 3D vision engineer

Responsibilities: turn camera, depth, or lidar data into object, geometry, and pose information for grasping, navigation, and interaction.

Common technologies: camera and hand-eye calibration, point cloud processing, segmentation, depth estimation, 6D pose estimation, OpenCV, PCL / Open3D, PyTorch, and ONNX / TensorRT deployment.

Interview preparation: camera models, coordinate transformations, reprojection error, point cloud registration, and pose evaluation; training data, occlusion, and lighting variation; and accuracy-latency tradeoffs in deployment. Readers with a CV background can build on existing visual modeling skills while learning geometry, sensors, and robot coordinate systems.

Related learning: Sensor calibration and sim2real · Sensor projection.

7.4 Engineering and platform roles

Simulation engineer

Responsibilities: build training and evaluation environments, assets, and scenes; configure physics and sensors; and support real-to-sim calibration, synthetic data, and automated evaluation.

Common technologies: Isaac Sim / Isaac Lab, MuJoCo, USD / URDF / MJCF, collision and contact modeling, actuator modeling, domain randomization, parallel simulation, and performance analysis.

Interview preparation: how mass, inertia, friction, joints, and time steps affect results; how physical measurements calibrate models; and how to diagnose performance bottlenecks and keep experiments reproducible. Roles involving video world models also require generative modeling, action-conditioned prediction, and evaluation. Check whether the emphasis is model research or simulation tooling.

Related learning: Simulation engineering.

Data engineer

Responsibilities: build teleoperation and collection workflows, data platforms, and quality checks that turn raw records into trainable, traceable, reusable datasets.

Common technologies: Python, device interfaces, video and time-series processing, data formats and metadata, object storage, scheduling, version management, and toolchains such as LeRobot.

Interview preparation: aligning observations and actions; detecting dropped frames, calibration errors, and unusual trajectories; splitting datasets, preventing leakage, improving loading throughput, and tracking versions. XDOF's robotics data engineer role emphasizes teleoperation data validation and quality tools as an example of these responsibilities.

Check the role's scope: collection operators emphasize equipment operation, task execution, and acceptance checks; teleoperation developers emphasize hardware, software, and control interfaces; data platform engineers emphasize storage, processing, quality, and training integration.

Related learning: Data engineering.

Robotics software / deployment engineer

Responsibilities: connect perception, policies, controllers, and hardware into systems that keep running, covering communication, inference services, device interfaces, logging, fault recovery, and releases.

Common technologies: C++, Python, Linux, ROS 2, DDS / QoS, multithreading, real-time scheduling, ONNX Runtime / TensorRT, and edge platforms such as Jetson Orin.

Interview preparation: C++ memory and resource management, concurrency and data races, communication timing, end-to-end and tail latency, CPU / GPU profiling, and handling disconnected devices, model timeouts, and incompatible versions. Prepare a system integration example that shows monitoring, diagnosis, and recovery to demonstrate engineering ability.

Related learning: Deployment engineering.

7.5 Hardware roles

Hardware roles draw on different disciplines and directly affect payload, precision, power consumption, and reliability.

Embedded systems engineer

Responsibilities: develop MCU firmware, sensor and actuator interfaces, real-time tasks, and low-level communication.

Technologies and interview preparation: C / C++, RTOSs, interrupts, DMA, CAN / CAN-FD, SPI / I²C; task scheduling, bandwidth and latency, watchdogs, error recovery, firmware updates, and diagnosing timing issues with an oscilloscope or logic analyzer.

Motor drives / electronics engineer

Responsibilities: motor drive work implements current, velocity, and position control; electronics work covers power supplies, control boards, sensor interfaces, and circuit reliability.

Technologies and interview preparation: field-oriented control (FOC), encoders, current sampling, PWM, power devices, schematics and PCBs, power supplies, EMC, and cooling. Explain control-loop bandwidth, sampling timing, measurement errors, and protection mechanisms, backed by measurements and failure analysis.

Mechanical / actuator / dexterous-hand engineer

Responsibilities: design links, joints, transmissions, actuators, and hand mechanisms, including component selection, assembly, and testing. Dexterous hands also involve finger actuation, tactile integration, and maintainability.

Technologies and interview preparation: CAD, mechanisms, materials, tolerances, finite element analysis, and motor and gearbox selection. Derive torque, speed, and travel requirements from a task, explain tradeoffs among stiffness, backlash, weight, cooling, and cost, and support the design with drawings, prototypes, and tests.

Related learning: Hardware.

7.6 Understanding role boundaries

Read the responsibilities and deliverables

Team size, product stage, and technical approach affect the division of work. An algorithm role in a small team may cover collection, training, and deployment; with more specialization, several teams may share these responsibilities. Check which robots and tasks the role handles, what it delivers, how it is evaluated, and whether it includes hardware work.

Hiring demand and role counts depend on location, time, and specific openings. The public postings above illustrate responsibilities and were consulted on September 24, 2026.

Distinguish two RL directions

Direction	Main goals	What to add to an RL background
Learning-based motion control	Balance, velocity tracking, terrain adaptation, and whole-body coordination	Dynamics, contact, simulation, actuators, and sim-to-real transfer
RL post-training for VLA / manipulation policies	Task success, execution efficiency, generalization, and failure recovery	Multimodal models, action representations, demonstrations, distributed training, and interaction-based evaluation

Readers with an RL background can try one motion control experiment and one manipulation policy experiment before choosing a direction. A VLA research review can demonstrate understanding of problems and methods; reproductions, arm manipulation, and hardware deployments add evidence of implementation and validation.

For a longer-term interest in products combining AI and hardware, look for projects that expose you to collection, training, deployment, and field failures. Explain the task, your own contribution, results, and failure analysis so your experience connects clearly with role requirements.

Continue learning

Return to 6. Technology stack for the relevant theory courses, then choose a project to connect the concepts in practice.