Pulmonary tuberculosis is a deadly infectious disease which occurs in many countries in Asia and Africa. In Indonesia, many people with tuberculosis disease are examined in the community health center. Examination of pulmonary tuberculosis is done through sputum smear with Ziehl-Neelsen staining using conventional light microscope. The results of Ziehl-Neelsen staining will give effect to the appearance of tuberculosis (TB) bacteria in red color and sputum background in blue color. The first examination is to detect the presence of TB bacteria from its color, then from the morphology of the TB bacteria itself. The results of Ziehl-Neelsen staining in sputum smear give the complex color images, so that the clinicians have difficulty when doing slide examination manually because it is time consuming and needs highly training to detect the presence of TB bacteria accurately. The clinicians have heavy workload to examine many sputum smear slides from the patients. To assist the clinicians when reading the sputum smear slide, this research built computer aided diagnose with color image segmentation, feature extraction, and classification method. This research used K-means clustering with patch technique to segment digital sputum smear images which separated the TB bacteria images from the background images. This segmentation method gave the good accuracy 97.68%. Then, feature extraction based on geometrical shape of TB bacteria was applied to this research. The last step, this research used neural network with back propagation method to classify TB bacteria and non TB bacteria images in sputum slides. The classification result of neural network back propagation are learning time (42.69±0.02) second, the number of epoch 5000, error rate of learning 15%, learning accuracy (98.58±0.01)%, and test accuracy (96.54±0.02)%.