Crop classification with deep convolutional neural network based on crop feature

Document Type : Original Article


Remote Sensing Center, Faculty of Earth Sciences, Shahid Beheshti University, Tehran, Iran


Given that agriculture has the most important role in ensuring food security (Johnston & Kilby,1989), it is necessary to prepare a map that shows the spatial distribution, land area, and type of crops cultivated with high accuracy (Cai et al., 2018). Agricultural land cover is relatively dynamic and variable at relatively short intervals. This makes it difficult to classify crops on satellite imagery (Bargiel, 2017). The lack or absence of ground truth data is another cause. Therefore, methods that are less dependent on ground samples and use phenological features derived from time series of bands and vegetation indices to classify crops will be more appropriate (Ashourloo et al., 2020). The purpose of this study is to use a deep learning method based on convolutional networks to classify the crop types and improve the performance of this network by using feature channels as an input image to the network and increasing the classification accuracy.
Materials and methods:
In this study, the visible and near-infrared bands of Sentinel-2 satellite on 10 different dates from 2019 for an area in Idaho, USA, as an important agricultural area, and the cropland data layer for extracting the crop types ground labels was used (Han et al., 2012). Then, in MATLAB software, the time series of spectral bands were constructed and using them, temporal profiles of NDVI for any crop were extracted to identify the unique phenological features of crops. Then, the functions developed based on the phenological characteristics of crops were applied to the time series of the bands and a feature channel was obtained for each crop that in two separate processes, once bands and once again feature channels were used as input to the CNN and the network was trained and the results of network performance on crop classification in the test site, were compared.
Results and discussion:
In the first stage, the time series of bands formed the input of the deep convectional neural network and the network was trained in the training area, using the tempo-spectral information of bands as the input channels and crops ground samples as the related labels. Due to the spectral overlap of the crops in some time periods, network training was associated with a relatively high loss and therefore, for the test area, the overall classification accuracy was 69% (percent) and the kappa coefficient was 0.55. In the next step, the functions that were developed as phenological features for crops were applied on the time series of the bands, and for each crop, a feature channel was obtained as the special feature of that crop. Then the algorithm was implemented using these feature channels in the test area and the overall accuracy was upgraded to 86% and the kappa coefficient to 0.82 compared to which indicated a significant improvement in the results compared to the previous case.
The deep convolutional neural network is very sensitive to the type of input channels for detecting agricultural crops and selecting the channels with suitable tempo-spectral characteristics for different types of crops, has a great impact on the accuracy of network training and can reduce the loss of training network and increase its efficiency in the classification of various crops.


Ashourloo, D., Shahrabi, H.S., Azadbakht, M., Aghighi, H., Matkan, A.A. and Radiom, S., 2018. A novel automatic method for alfalfa mapping using time series of landsat-8 OLI Data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 11(11), 4478-4487.
Ashourloo, D., Shahrabi, H.S., Azadbakht, M., Aghighi, H., Nematollahi, H., Alimohammadi, A. and Matkan, A.A., 2019. Automatic canola mapping using time series of sentinel 2 images. ISPRS Journal of Photogrammetry and Remote Sensing. 156, 63-76.76.
Ashourloo, D., Shahrabi, H.S., Azadbakht, M., Rad, A.M., Aghighi, H. and Radiom, S., 2020. A novel method for automatic potato mapping using time series of Sentinel-2 images. Computers and Electronics in Agriculture. 175, 105583.
Azzari, G. and Lobell, D.B., 2017. Landsat-based classification in the cloud: an opportunity for a paradigm shift in land cover monitoring. In: Remote Sensing of Environment, Big Remotely Sensed Data: tools, applications and experiences. 202. pp. 64–74. https://
Bargiel, D., 2017. A new method for crop classification combining time series of radar images and crop phenology information. Remote sensing of environment 198, 369-383.
Boryan, C., Yang, Z., Mueller, R. and Craig, M., 2011. Monitoring US agriculture: the US department of agriculture, national agricultural statistics service, and cropland data layer program. Geocarto International. 26(5), 341-358.
Cai, Y., Guan, K., Peng, J., Wang, S., Seifert, C., Wardlow, B. and Li, Z., 2018. A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote sensing of environment. 210, 35-47.
Esch, T., Metz, A., Marconcini, M. and Keil, M., 2014. Combined use of multi-seasonal high and medium resolution satellite imagery for parcel-related mapping of cropland and grassland. International Journal of Applied Earth Observation and Geoinformation. 28, 230-237.
Foerster, S., Kaden, K., Foerster, M. and Itzerott, S., 2012. Crop type mapping using spectral–temporal profiles and phenological information. Computers and Electronics in Agriculture. 89, 30-40.
Gadiraju, K.K. and Vatsavai, R.R., 2020. Comparative analysis of deep transfer learning performance on crop classification, Proceedings of the 9th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, pp. 1-8.
Gholampur, A., 2008. A novel algorithm for detecting wheat and barley. M.Sc. Thesis. Shahid Beheshti University, Tehran, Iran.
Goodarzdashti, S., 2021. Automatic crop detection based on phenological information using Google Earth Engine. M.Sc. Thesis. Shahid Beheshti University, Tehran, Iran.
Huang, B., Zhao, B. and Song, Y., 2018. Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery. Remote Sensing of Environment. 214, 73-86.
Johnston, B.F. and Kilby, P., 1982. Unimodal and bimodal strategies of agrarian change. Rural development: theories of peasant economy and agrarian change. London: Hutchinson Publishing Group.
Kang, Y., Khan, S. and Ma, X., 2009. Climate change impacts on crop yield, crop water productivity and food security – A review. Progress in Natural Science 19, 1665-1674.
Khatami, R., Mountrakis, G. and Stehman, S.V., 2017. Mapping per-pixel predicted accuracy of classified remote sensing images. Remote Sensing of Environment 191, 156-167.
King, L., Adusei, B., Stehman, S.V., Potapov, P.V., Song, X.-P., Krylov, A., Di Bella, C., Loveland, T.R., Johnson, D.M. and Hansen, M.C., 2017. A multi-resolution approach to national-scale cultivated area estimation of soybean. Remote Sensing of Environment 195, 13-29.
LeCun, Y., Bengio, Y. and Hinton, G., 2015. Deep learning. Nature. 521, 436-444.
Mingwei, Z., Qingbo, Z., Zhongxin, C., Jia, L., Yong, Z. and Chongfa, C., 2008. Crop discrimination in Northern China with double cropping systems using Fourier analysis of time-series MODIS data. International Journal of Applied Earth Observation and Geoinformation. 10, 476-485.
Peña-Barragán, J.M., Ngugi, M.K., Plant, R.E. and Six, J., 2011. Object-based crop identification using multiple vegetation indices, textural features and crop phenology. Remote Sensing of Environment. 115, 1301-1316.
Ramankutty, N., Mehrabi, Z., Waha, K., Jarvis, L., Kremen, C., Herrero, M. and Rieseberg, L.H., 2018. Trends in global agricultural land use: implications for environmental health and food security. Annual review of plant biology. 69, 789-815.
Sandborn, A., Mueller, R., Boryan, C., Johnson, D., Yang, Z., Ebinger, L., Rosales, A., Willis, P., Seffrin, R., Jennings, R., Deaton, M. and Hamer, H., 2019. NASS Geospatial Applications from the Cropland Data Layer. ISI World Statistics Conference, Malaysia, Aug 18-23, 2019. Posted 8/21/2019.
Shelestov, A., Lavreniuk, M., Kussul, N., Novikov, A., Skakun, S., 2017. Exploring Google Earth Engine Platform for Big Data Processing: Classification of Multi-Temporal Satellite Imagery for Crop Mapping. Front. Earth Sci. 2017, 5, 17.
Zheng, B., Myint, S.W., Thenkabail, P.S., Aggarwal, R.M., 2015. A support vector machine to identify irrigated crop types using time-series Landsat NDVI data. International Journal of Applied Earth Observation and Geoinformation 34, 103-112.
Zhong, L., Hu, L., Zhou, H., 2019. Deep learning based multi-temporal crop classification. Remote sensing of environment 221, 430-443.
Zhong, L., Hu, L., Zhou, H., Tao, X., 2019b. Deep learning based winter wheat mapping using statistical data as ground references in Kansas and northern Texas, US. Remote Sens. Environ. 233, 111411.
Xu, J., Zhu, Y., Zhong, R., Lin, Z., Xu, J., Jiang, H., Huang, J., Li, H., Lin, T., 2020. DeepCropMapping: A multi-temporal deep learning approach with improved spatial generalizability for dynamic corn and soybean mapping. Remote Sensing of Environment 247, 111946.