Can Permanent magnets lose their magnetism

Yes, Permanent magnets can lose their magnetism. There are three main causes which can affect the magnetism of a permanent magnet.

They are

Heating a magnet above the Curie Temperature (the temperature above which the magnetic properties of a material change from ferromagnetic to paramagnetic) causes the magnetic domains to be disrupted permanently. Mild heating causes a reduction in the magnetism. However, when it cools the full magnetism is restored.

Mechanical Shock:
A magnet that is subjected to shock such as being hit by a hammer or dropped from a height can lose its magnetism. However, modern magnets made from materials such as Samarium Cobalt and Neodymium can withstand shock.

An opposing magnetic field:
A demagnetising field or a field that acts in the opposite direction can also result in a loss of magnetism. Demagnetising fields are sometimes used to reduce the strength of a magnet to fit a specific application