Specific instability of boreholes drilled through rocks, such as borehole breakout, borehole collapse, and shale swelling, is a significant concern in drilling operations due to the adverse impact it has on drilling safety and efficiency. Despite considerable research on borehole instability, finding a solution to this problem remains elusive. The objective of this review is to examine the primary mechanisms of instability in rocks from a general physical instability principle perspective. A key criterion for the analysis of specific instability problems is the construction of comprehensive phase diagrams. Within this context, three main approaches and their partial implementation are discussed. The first approach involves constructing phase diagrams under the assumption that the borehole has already caved and aims to determine the final position of the cave wall. However, this approach presents several challenges. Caving may lead to severe kick-off or break-off problems, instability during the caving process, and complications in simultaneously modeling caving in a fluid-like porous material alongside flow filtration through the open walls of the borehole. The second approach entails developing phase diagrams that characterize the mechanical and hydraulic instability of the borehole wall being driven. These phase diagrams serve as specific instability criteria, but they face difficulties in generalizing across various failure mechanisms since they are typically non-quasi-static and are disregarded after failure occurs and the propagation ban is implemented. The third approach involves analyzing phase diagrams to investigate the results of the touch modes of compliance. However, these phase diagrams and the collapse of final boundary conditions often overlook the main depletions and in-situ impedances, which are crucial system-specific physical complements that enhance the classical mode of balance equation. Therefore, the integration of these factors is essential for a more comprehensive understanding of borehole instability.