Most racetracks of the 20th Century were built with a similar theme. You had a large grandstand, with rows and rows of seats, not unlike a baseball or football stadium, usually located above more comfortable box seats that are occupied by owners or trainers. There was usually a glass-enclosed dining room or turf club up high, segregating the upper crust from the $2 gamblers in the floors below.
On the far side of the track you’d often find a spacious barn area and the remainder of the perimeter was parking: lots and lots of parking for the large crowds that frequently came out to the races.
Many of these tracks were built in low-density population areas outside of major cities. As the cities expanded (or many of its residents opted to move into the suburbs), the land on which these 20th Century racetracks were built became more valuable. At the same time, as the sport faced increased competition and convenience betting allowed horseplayers to watch and wager on races from just about anywhere, those once-large racetrack crowds began to shrink.