Explain the difference between L1 and L2 regularization in the context of regression analysis, and why we might choose one over the other?

Interview question asked to Data Scientists interviewing at Soundcloud, AppDynamics, Nutanix and others: Explain the difference between L1 and L2 regularization in the context of regression analysis, and why we might choose one over the other?.