Early detection of colorectal cancer in medical imaging is important for improving patient outcomes and guiding treatment strategies. In order to classify the colon cancer, this study proposes the automated image analysis system that integrates deep learning driven classification. The Adaptive Non-Local Means (ANLM) filter is initially employed to denoise the colon cancer images while preserving fine structural and textural characteristics. To enhance border consistency and region homogeneity, spatially constrained K-means clustering is used to accurately segment the malignant areas by leveraging spatial neighborhood information. From the segmented areas, Gray Level Co-occurrence Matrix (GLCM) is applied to extract the discriminative textural features, capturing vital statistical information regarding tissue heterogeneity and morphological changes. The dataset named Lung and Colon Cancer Histopathological Images dataset is used which is available in kaggle. An Attention-Guided Residual Transformer Network (AGRT-Net), which blends residual learning with self-attention processes to model long-range dependencies and highlight diagnostically relevant aspects, is then used to classify these features. By focusing on cancer patterns, the proposed model improves the feature discrimination. Using the Python software, the AGRT-Net achieves higher uniform value of 0.979 across the accuracy, precision, recall, and F1-Score, which demonstrates robust potential for computer-aided colon cancer diagnosis.