The-photograph-stands-alone crowd notwithstanding, every photograph "published" needs a caption. In this case our definition of "published" is very loose. It includes photographs posted to social media, photo sharing sites, "professional portfolio" sites, books (either print or digital), magazines (either print or digital)... just about anywhere really. And in consideration of the-photograph-stands-alone crowd, we often get something approaching a caption in the exhibit catalog.
The purpose of the caption is to give the reader information that puts the photograph in context, provides additional information that is not readily available from the photograph and to amplify and extend the reader's understanding of the photograph. Remember that photography is part of a complex communication system and that the most effective communication of the message or story happens when the photographs and the words work together. Without properly written captions the reader is left to flail about to come up with some level of significance for the photograph.
Captions are astonishingly deceiving. They are short and because they are short folks feel that are easy to write. Often we get a caption that simply restates the photograph. This is especially true if it starts out "pictured here..." or ends with "poses for a photograph". And then there are the captions that drift off into discussions of what the photographer went through to "get the photograph" or the equipment and various settings that were used to "get the photograph". None of these discuss the context of the photograph or fulfill the function of the caption.