Quantification and assessment of disease symptoms are important elements of plant disease management systems and are required to assist with making decisions on the choice of protective agents to be applied to crops or for screening plant genotypes for the development of resistant varieties. Traditional methods of identifying and quantifying disease severity are cumbersome, involving visual assessment tools or scales, and rating of plants at a point in time. Visual assessment is prone to human bias and error, thereby reducing the efficiency and accuracy of this method. In this study, we developed a smartphone camera-based image recording, processing, and assessment tool for measurement of symptoms of early and late blight, and bacterial leaf spot diseases in sweet pepper caused by Alternaria solani, Phytophthora infestans, and Xanthomonas campestris pv. Vesicatoria, respectively. Sweet pepper or bell pepper is a major vegetable crop grown in the Caribbean region, but production is severely affected by plant diseases, most important of which include foliar infections by fungi and bacteria that cause major losses in fruit yield. This research utilized smartphone captured images of leaf specimens for severity measurement and classification of diseases. The steps involved were color space conversions, detection of leaf area by Otsu’s method, and thresholding for foliar diseased area detection and quantification. Gray-Level Co-occurrence Matrix (GLCM) extracted the texture features from the diseased area of leaves. These features are trained and classified by various machine learning classifiers including trees, rule-based and Bayes models. Application of decision trees and rule-based classifier models achieved 98% accuracy individually, while Bayes model achieved 86% accuracy. The image input into the above classifier models resulted in fast and accurate identification of the diseases by matching the features of trained images of disease symptoms. This method could work well for leaves collected from field-grown plants as well as from inoculated greenhouse plants.