The goals of "smart cities" include relieving the burden of growing urbanisation, cutting down on energy use, protecting the environment, boosting the local economy and people's standard of living, and facilitating wider access to and use of cutting-edge Information and Communication Technologies (ICT). In smart cities, ICT is essential for policymaking, decision making, plan execution, and the delivery of useful services. The primary goal of this analysis is to investigate the part that AI and machine learning play. Examples of ed tech include Deep Reinforcement Learning (DRL) and Machine Learning (ML). In a complicated smart city setting, the aforementioned methods can be used to develop the best possible rules. Smart transportation, cyber-security, energy-efficient usage of smart grids (SG), efficient use of Unmanned Aerial Vehicles (UAVs) to guarantee the best 5G and beyond 5G (B5G) communications, and a smart health monitoring are all discussed in length in this paper. Finally, we discuss the research challenges that have yet to be met and potential future research directions that could bring the concept of a "smart city" closer to fruition.