From Imitation Learning to Online Reinforcement Learning for Industrial 3D Bin Packing

This thesis addresses the industrial 3D Bin Packing Problem (3D-BPP) by investigating learning-based approaches for smart palletizing under realistic operational constraints. Unlike classical formulations that focus mainly on geometric packing efficiency, industrial palletizing must also consider constraints related to mechanical stability, load distribution, stackability, and robotic execution feasibility. To capture this complexity, the palletizing process is modeled as a sequential decision-making problem within a learning framework. The proposed methodology follows a progressive pipeline from imitation learning to reinforcement learning. First, a policy is trained through behavioral cloning using expert pallet layouts generated by an industrial optimization algorithm, allowing the model to learn structured packing strategies from validated solutions. Building on this baseline, offline reinforcement learning is explored to improve policy performance using fixed datasets. Due to the limitations observed in this setting, the approach ultimately evolves toward an online reinforcement learning framework based on Proximal Policy Optimization (PPO), where the agent learns directly through interaction with a simulated environment. To support efficient learning while preserving physical realism, the palletizing environment is first represented using a voxel-based discretization and later extended to a continuous formulation enabling more precise placements. The framework also includes mechanisms for multi-pallet scenarios and industrial feasibility constraints. The proposed approach is evaluated across multiple scenarios and validated through physics-based simulation in NVIDIA Isaac Sim, showing that the learned policy can generate stable and efficient palletizing layouts compatible with realistic industrial environments.