A leaving group , LG, is an atom (or a group of atoms) that is displaced as stable species taking with it the bonding electrons. Typically the leaving group is an anion (e.g. Cl^-) or a neutral molecule (e.g. H₂O). The better the leaving group, the more likely it is to depart.
 

What do we mean by this ? First we should write the chemical equations for the two processes:
 

These two equations represent Bronsted acid dissociation and loss of a leaving group in a S_N1 type reaction.
Note the similarity of the two equations: both show heterolytic cleavage of a s bond to create an anion and a cation.

For acidity, the more stable A^- is, then the more the equilibrium will favor dissociation, and release of protons meaning that HA is more acidic.

For the leaving group, the more stable LG^- is, the more it favors "leaving".

Hence factors that stabilize A^- also apply to the stablization of a  LG^-.

Here is a table classifying some common leaving groups that we will eventually meet......
 

Note, once again, that HO^- is a poor leaving group (remember from the reactions of alcohols ?).... after all it is the conjugate base of water.... and when we turn on a tap in the kitchen, we aren't usually trying to get a strong acid to drink !
But water itself, H₂O, is a good leaving group, since it is the conjugate base of H₃O⁺, which is a strong acid.