The glue that binds rubbly asteroids...

isn't gravity: Small Asteroids Are Held Together by Van Der Waals Forces.