The museum building was originally arailway station, Gare d'Orsay, constructed for the Chemin de Fer de Paris à Orléansand finished in time for the 1900 Exposition Universelle to the design of three architects: Lucien Magne, Émile Bénardand Victor Laloux. It was the terminus for the railways of southwestern France until 1939.
By 1939 the station's short platforms had become unsuitable for the longer trains that had come to be used for mainline services. After 1939 it was used for suburban services and part of it became a mailing centre during World War II. It was then used as a set for several films, such as Kafka's The Trial adapted by Orson Welles, and as a haven for the Renaud–Barrault Theatre Company and for auctioneers, while the Hôtel Drouot was being rebuilt.
In 1970, permission was granted to demolish the station but Jacques Duhamel, Minister for Cultural Affairs, ruled against plans to build a new hotel in its stead. The station was put on the supplementary list of Historic Monuments and finally listed in 1978. The suggestion to turn the station into a museum came from the Directorate of the Museums of France. The idea was to build a museum that would bridge the gap between the Louvre and the National Museum of Modern Art at the Georges Pompidou Centre. The plan was accepted byGeorges Pompidou and a study was commissioned in 1974. In 1978, a competition was organized to design the new museum. ACT Architecture, a team of three young architects (Pierre Colboc, Renaud Bardon and Jean-Paul Philippon), were awarded the contract which involved creating 20,000 sq. m. of new floorspace on four floors. The construction work was carried out by Bouygues. In 1981, the Italian architect, Gae Aulenti was chosen to design the interior including the internal arrangement, decoration, furniture and fittings of the museum. Finally in July 1986, the museum was ready to receive its exhibits. It took 6 months to install the 2000 or so paintings, 600 sculptures and other works. The museum officially opened in December 1986 by then-president, François Mitterrand.
The square next to the museum displays six bronze allegorical sculptural groups in a row, originally produced for the Exposition Universelle (1878):