It‘s absolutely dangerous.
If a car swerved into its lane it would take evasive action and keep going. Tapping the gas pedal doesn‘t disable braking unless you keep pressing down on it. Tapping the brake would turn off Autopilot and he‘d have to instantly take over. There is an auditory alarm when autopilot turns off by itself, though I‘m not sure this guy would wake up from it.
In my experience when Autopilot shuts off unexpectedly because it doesn’t know what to do, you have to take over immediately, but don‘t hold me to that. I‘m normally attentive when using it so I‘d take over immediately. When there‘s no user input (applying torque to the steering wheel) after three warnings, the car will slow down to come to a halt and turn on the emergency indicators.