Abstract : Computer Vision and its associated emerging technology prove high potential in advanced agricultural applications. Recently Deep Learning algorithms are much more efficient in producing solution to Computer Vision Problems. In this paper, a review on deep learning algorithms and its uses in agriculture is done. Also, this paper reports the research work on autonomous harvesting system using IOT Technology. Detection of Cashew fruit is an important part of intelligent management in the Cashew harvesting process. For fast and accurate Cashew fruit detection in the complex environment, this paper proposes a fruit detection method using Deep Learning algorithms. We applied SSD mobilenetV2 neural network to get the features of cashew fruits in depth and detect different stages of cashew apple in tree like Immature, Unripe Cashew and Ripe Cashew. This method achieved an average of 92.26% detection rate and the average execution time is 8 seconds. Also, the results are compared with machine learning algorithm and YOLOv4 method. Deep learning method gives better results on accurate detection of cashew fruits and detection time when compared to machine learning algorithm. Compared with YOLOv4, Single shot Detector (SSD) had higher detection rate and confidence. The proposed method detects different colors of Cashew fruit like yellow, red and combination of both and also detects different maturity levels with high recognition rate and in faster speed. This technique also able to detect individual cashew fruit in clusters which effectively helps the harvesting robot for continuous fruit plucking process and for yield estimation. At the end, several problems related to recognition and localization are addressed.