Yes, that does make sense. The Crumble maintains the USB connection while connected to the PC. I.e. it doesn’t use a boot loader. It has a little virtual machine that runs your program, the VM and USB routines are polled in turn, so if the USB connection is not present that function will return quicker.
Not sure if you can fix it at the moment. I am working on a new version of the VM which will hopefully include servo outputs though.