St. Albans's building enterprise lay chiefly in St. James's Field. That is, east of St. James's Street. West of that street he had his leasehold interest until 1691, but little building seems to have been done here under his influence. The creation of the Green Park in 1668 and the consequent diminution of the Pulteney estate led to a rearrangement of the ownership of the land west of St. James's Street, so that, by 1697, most of it had ceased to belong to the Crown and had passed to private owners. It was on these private freeholds that new buildings were erected, for the most part by the speculative family business run by the Rossingtons. Small houses were built in the secluded new streets off St. James's Street, but larger ones went up on the only part of the Pulteney estate remaining in the possession of the Crown overlooking Green Park. During the eighteenth century encroachments on the park were made to give the present enfiladed line, which was established by 1798. The Crown has recently acquired land bordering the park further north. This intricate tenure of the area from Cleveland Row northward has probably helped to preserve its somewhat old-fashioned and retired character.

Thus the areas east and west of St. James's Street exhibit in their history a difference that is perceptible in their fabric. As has been seen, from 1665 the area eastward of St. James's Street was itself divided between the Crown estate and St. Albans's freehold. One or two other parts were disposed of by the Crown fairly soon, in the 1670's-at the north-east end of St. James's Street and Nell Gwynne's house in Pall Mall. Since then the boundary between Crown and private freehold has not been greatly altered.

Part of the St. Albans's freehold interest still survives, however, vested in the owner of the site of the eighteenth-century shop at No. 3 St. James's Street. It may be noted that this ownership maintains the Suffolk connexion introduced by St. Albans himself.

Very little of the first domestic building remains externally visible in the area. On most of the sites, indeed, (particularly those on Crown land) the present structure represents the third or fourth building. Notably on the south side of Pall Mall, but also elsewhere, this has resulted in a confusing history of the merging and division of building sites. Apart from small partially reconstructed terrace-houses, chiefly on freehold property in Jermyn Street and west of St. James's Street, and the reinstated façade of Schomberg House, the only seventeenth-century building of any consequence remaining is the church of St. James. Extensive rebuilding took place in the eighteenth century, doubtless from a desire for greater modishness and convenience rather than from structural necessity.

The plan on which the area was laid out gave an orderly, dignified and convenient pattern for a fashionable but not wholly exclusive neighbourhood. At its centre was the enclosure of the 'piazza' (St. James's Square) with its three centrally entrant streets. The placing of the church was clearly related to the square. But the layout was by no means dominated by a commanding desire for impressiveness at all costs. In construction, the houses in the square were more numerous and rather less palatial than first intended, and the plan of the square somewhat less symmetrical. The church as built did not fill its scenic rôle quite to the satisfaction of later critics, and the alignment of streets was occasionally impaired or distorted by adjustment to practical difficulties or the requirements of the ordinary commerce of London life.