Imagine, if you would, a set of speakers. They're designed to transmit sounds up to a particular level and within a particular range. Sounds that fall outside that range can permanently damage the speaker's internal components, creating a range of audio distortions or causing them to stop working entirely.
It may not be a perfect analogy, but your ears work in a similar fashion. Sound waves are first channeled through the ear canal, striking the eardrum, which then amplifies them through a series of tiny bones known as the ossicles. These bones then transmit the vibrations from the sound through a fluid-filled, snail-shaped organ known as the cochlea.
Tiny hairs within the cochlea, known as the stereocilia, pick up the vibrations from the fluid, translating them to electrical impulses, which are then sent to the auditory nerve and transmitted to the brain.
As you've probably guessed, the organs involved in this process are all incredibly delicate. In addition to wearing down due to age, they can also be permanently damaged if subjected to noise they weren't designed to transmit. This most commonly results in damage to the stereocilia, but in severe cases may also damage the ossicles or even the auditory nerve.