The chapter states that 'Names of cities' are categories of nouns. Rajshahi is a name of a city, hence a noun.
Class 6 › English Grammar and Composition › Chapter 1: Parts of Speech › Topic: Introduction to Parts of Speech
Aligned to the NCTB national curriculum.