in figurative feeling of "withdraw a charge," 1859, United states English, from notion of descending a ladder, etc.; from back (v.) + down (adv.).
remove oneself from an obligation
- move backwards from a certain place
I looked forward to weekends when he'd come back down to Boston and we'd get together.