A Review of Prevalent Methods for Automatic Skin Lesion Diagnosis

2.1. Epidermis

The epidermis is a layered scale-like tissue which serves as a protection against external belligerences (extreme radiation, wounds and contaminations). The epidermis consists of four types of cells, which are Keratinocytes, Melanocytes, Langerhans and Merkel cells.

2.2. Dermis

The dermis is composed of collagen and elastic fibres. The dermis has two primary sub-layers which are the Papillary dermis (thin layer) that acts as a glue to hold the epidermis, and the dermis and Reticular dermis (thick layer) that supplies energy and nutrition to the epidermis. It contains nerve endings, sweat glands, hair follicles, blood vessels and lymph vessels. In addition, it is responsible for healing and sense of touch.

2.3. Subcutaneous

The subcutaneous layer is responsible for supplying nutrients to the other two layers. The subcutis, being made of made of fat and connective tissue that helps to cushion and insulate the body.

6.1. Skin Lesion Image Acquisition and Preprocessing

Results of diagnosis reported in the literature have been judged to be highly dependent on the volume and quality of images used [29, 102, 103]. Often, variations in devices used in capturing lesion images and conditions under which these images are acquired have been observed to adversely affect the results of automated skin lesion diagnosis. In the time past, the source of image data for lesion screening was colour slides. However, over the past decades, it has been proven that quality and accurate diagnosis can be achieved using digitized lesion images [104-106]. The two predominant dermatological image types are macroscopic (clinical) and microscopic (dermoscopic) images. While the use of digitised dermoscopic images is on the increase, some reports have argued that pertinent distinguishing image features (diminishing textures and pored) are easily examined using macroscopic images rather than under dermoscopic images [107].

The literature has reported several imaging techniques that could assist in the acquisition and screening of skin lesion images [108-110]. One of such popular technique is dermoscopy which provides in-vivo, non-invasive imaging of skin lesion using different light magnification systems [14, 17, 45, 52, 56, 58-61, 66, 67, 111-131]. Other notable imaging techniques include digital photography [108, 110, 131-134], radiography [110, 135], confocal microscopy [80, 108, 115, 133, 136-148], tomography such as computed tomography, positron emission tomography, photoacoustic tomography, optical coherence tomography and magnetic resonance imaging [108, 110, 149-172], ultrasound imaging [108, 110, 173-180], multispectral imaging [108, 181, 182] and thermal imaging (thermography) [183-186]. A review of non-invasive imaging techniques was recently discussed by Menge & Pellacani [109], detailing the application of various imaging techniques and the accompanying shortfalls. Arguably, due to slow adoption of advances in diagnostic technology by many dermatologists, the trend noticed in the literature is a growing increase of the usage of dermatoscopy (Dermoscopy). Recently, the usage of dermatoscope with mobile phone camera has also been discussed in some studies for making acquisition of lesion images easier [28, 187, 188]. Reflectance microscopy has equally been dubbed to give good result against the light coloured melanoma lesions [80].

While each individual imaging method has produced a promising result in the screening of lesions, there has been a rise in the mixture of imaging methods to enhance sensitivity, specificity and accuracy of lesion screening [151, 152, 189, 190]. This is further validated by Mohr et al. [191] and Reinhardt et al. [153] and recently by Bourgeois et al. [170] that the combination of Positron Emission Tomography and Computed Tomography (PET/CT) revealed a better sensitivity in staging of malignant tumours. Wang et al. [173] equally argued that integrating photoacoustic tomography with ultrasound has yielded a better specificity when compared to when either method was used in isolation. The combination of confocal and photo thermal microscopy was recently discussed by He et al. [192] for noninvasive and label free 3-D imaging of melanoma. A good review was conducted by Dancey et al. [110] to compare various techniques used in imaging melanoma, and consequently recommended a choice of imaging techniques based on their applicability, accuracy and cost. In the review [110], it has been suggested that ultrasound imaging (ultrasonography) is the most effective mode of screening in the absence of sentinel lymph node biopsy. A similar view was shared by Xing et al. [189] during the comparison made between the usage of ultrasonography, CT, PET and PET/CT in staging and surveillance of melanoma patients.

In image processing, commonly used colour spaces include Red-Green-Blue (RGB and sRGB), Commission Internationale de l'Eclairage (CIE L*a*b, CIE L*u*v and CIE X*Y*Z), Luma plus chrominance (Y’CbCr, Y’PbPr, Y’UV and YIQ) and Hue-Saturation-Intensity-Value-Luminance (HSI, HSV/B and HSL). Most digitized lesion images are commonly generated as RGB. However, because of device dependency of RGB colour space, digitized lesion images are often converted to greyscale or blue channel for single channel (scalar) processing in order to represent the intensity of the image. In a bid to ease the accuracy of classification, Dobrescu et al. [48] converted each image used in their study to 256 grey levels image of the same size as a form of preprocessing of the image in Hue Saturation Value (HSV) colour space. Multichannel (vector) processing can equally be used to take advantage of the original colour information of the lesion. The main challenge, however, with the use of vector images is the computational requirement. Gómez et al. [193] argued that it is implausible that a particular colour space is optimal across different dermoscopic images acquired via different systems, even though the images have similar prognosis. Some reports [194-196], however, revealed that CIE L*a*b colour space produced a convincing result compared to its counterparts (CIE L*u*v and CIE X*Y*Z) and the popular YCbCr colour space when performing preprocessing of multichannel microscopic lesion images.

The term preprocessing in lesion image diagnostic procedures usually encompasses lesion image enhancement, image restoration with neighbourhood pixels and artefact removal [197]. The conditions surrounding the acquisition of lesion images generally influence possible discriminating features that can be extracted from such images for the purpose of automated diagnosis. Rahman et al. [198] reasoned that retrieval and the classification tasks of lesion could be challenging when images collected from separate data sets are captured by different devices under varying conditions (such as lightening). This creates a non-uniform illumination pattern, thus confusing diagnostic procedures. Colour calibration of image acquisition device has been one of the approaches proposed in the literature to resolve such challenges [199-205]. Low contrast of lesion images could also make isolation of lesion a very difficult task [206]. Abbas et al. [195] proposed enhancing lesion image contrast by adjusting and mapping the intensity values of the lesion pixels in the specified range in CIE L*a*b colour space. One major flaw of contrast enhancement is over amplification of noise in the region having relatively small intensity range. The use of Contrast Limited Adaptive Histogram Equalization (CLAHE) might be applied to address such limitations [207, 208]. (Figs. 2 and 3) respectively illustrate the normal and filled histogram of the image shown in (Fig. 4 and 5) shows an equalized histogram of the same image in (Fig. 4) for better noise removal resolution.

A major hindrance to a successful diagnosis in medical skin imaging is the presence of artefacts, typically referred to as noise. Artefacts such as hair shaft (Figs. 6 and 7), dermoscopic gels, thin blood vessel, shadows, ruler marking, specular reflections, vignetting and air bubble can confuse diagnosis and impede achievement of better accuracy in automated diagnosis process [107, 209-211]. To resolve the challenges posed by these artefacts, the literature report the use of a number of approaches which consist primarily of artefact detection (Fig. 5) and subsequent artefact removal (Fig. 4). Methods used for aiding the detection of artefact include filtering (curvilinear matched, Prewitt, Gaussian, median and bilateral), derivative of Gaussian, morphology operations (closed based top hat) and anisotropic diffusion. Filtering is a popular method to smooth a lesion image before detecting artefacts. Bilateral filtering has been seen to perform very well amongst other types of filtering because of its edge-preserving smoothing operation on the lesion images, especially on microscopic images [194]. Karkunen-Loéve is another method often used to preserve artefact edges during image smoothening. Prominent among the artefact removal methods is the linear interpolation [212, 213]. This was popularized in the demonstration of the system named DullRazor that was proposed by Lee et al. [213] to remove hair artefacts from a given lesion image. Other commonly used artefacts removal methods include inpainting (partial differential equation, exemplar-based, fast marching) [214-219] and region growing [107, 220]. A promising method called lacunarity algorithm which is a measure of transitional invariance for computing aspects of patterns exhibiting scale-invariant changes in the structure was equally proposed by Gilmore et al. [96] to avoid the need for a more sophisticated method.

The hair shafts and ruler marking appear to be the most common artefacts reported in the literature [210, 214, 221-226]. In our study, we observed that much effort has been given to the removal of hair shaft and ruler markings from lesion images [107, 209, 210, 213, 214, 220, 225, 227]. An excellent review by Abbas et al. [55] discussed a comparative study of the state-of-the-art algorithms for automatic detection of hair and restoration, vis-à-vis their applicability to the texture-part of lesion images. A novel algorithm comprising of morphological and fast marching schemes was also suggested in a study [55]. Similar procedure of using fast marching inpainting was discussed by Okuboyejo et al. [194] towards improving the speed of preprocessing of dermoscopic lesion images. Toossi et al. [228] also suggested the usage of multi-resolution coherence transport inpainting based on wavelet-based structure for the removal of hair artefacts in dermoscopic images. The algorithm proposed in a study [228] combines simple coherence transport inpainting with a wavelet decomposition and reconstruction method in an iterative and multi-resolution structure.

6.2. Lesion Image Segmentation

The successful segmentation of skin lesion from the healthy surrounding skin is a pertinent requirement for a workable lesion diagnostic process. The analysis of a number of the dermoscopic features (asymmetry, border sharpness) and clinical features (asymmetry, border irregularity) is only as accurate as the estimated lesion boundary. The variations in human interpretation of manual lesion boundary tracing have equally influenced the automation of lesion segmentation procedure [229, 230]. According to the literature, the estimation of lesion border by dermatologists has been reported to depend upon higher-level knowledge, leading to poor reproducibility of segmentation results [231]. However, Silletti et al. [232] argued that with exception of the Fuzzy C-Means (FCM), some state-of-the-art automatic segmentation methods performed poorly when compared with segmentation carried out by expert dermatologists. In Fig. 8), an example is shown on a segmented lesion image that has been localized from its surrounding healthy skin.

The segmentation task has sometimes been referred to as one of the most difficult tasks in medical imaging. Among other concerns, the difficulty can be attributed to low-contrasts surrounding the skin, fuzzy borders, the existence of artefacts and irregular structures characterizing lesion images [48, 65, 211, 233-235]. Readers can refer to the previous section detailing preprocessing techniques for image contrast enhancement and removal of occluding artefacts typically found in both macroscopic and microscopic images. Some reports in the literature have equally suggested that tumour areas manually extracted by dermatologists have been discovered to be sometimes characterized with inconsistency [232, 236-238], validating the need for an automated lesion segmentation approach that can aid reproducibility of results. In recent times, the literature has seen a great improvement in automating lesion image segmentation from the surrounding healthy skin parts for the purpose of achieving automated diagnosis of such lesion images. However, Chang et al. [239] argued that it is impractical to perform fully automatic segmentation on all skin lesion images due to reasons such as complexities surrounding acquisition of lesion images.

Most segmentation approaches incorporate some forms of image preprocessing to reduce or eliminate image noises such as air bubbles, ruler marking, hair shafts that could confuse segmentation. An example of this is the application of combined spline and B-spline by Abbas et al. [240] to enhance the quality of dermoscopic images before segmentation. The Karkunen-Loéve Transform (KLT) also known as Principal Component Analysis/Transform (PCA/T) was used to enhance the edges of the lesion image for better segmentation result in some studies [20, 57, 193, 241, 242]. The top-hat and bottom-hat transformations were applied in a study [243] to maximize the contrast of lesion images in order to achieve a comparable lesion segmentation using ensemble methods. The literature has chronicled the numerous lesion localization (border detection) approaches that can help to segment pigmented skin lesion from the neighbouring region in an automated mode. A number of lesion segmentation algorithms (including edge based, region based and thresholding) have equally been proposed in the literature. In the course of our study, we observed that most of the reported segmentation methods in the literature are based on the colour information of the lesion being examined arguably due to the simplicity of the representation of lesion colour properties. Some reports [244-249] have equally used texture properties of skin lesions to estimate lesion boundaries. Commonly adopted texture feature methods used in segmenting skin lesion areas include Grey Level Co-occurrence Matrix (GLCM) [245], Gabor functions [248], Laws texture energy masks [54], Markov Random Field (MRF) models [246, 250]. Glaister et al. [244] equally proposed a texture oriented lesion segmentation algorithm called Texture Distinctive Lesion Segmentation (TDLS). The TDLS algorithm uses joint statistical information to characterise skin and lesion textures as representative texture distributions. Maeda et al. [251] combined colour and texture features in a proposed Fuzzy-based hierarchical algorithm to achieve a perceptual segmentation of dermoscopic lesion images.

The edge based segmentation methods essentially use metadata about edges of a given lesion image in addition to related post-processing techniques to estimate the boundary of a lesion [219, 234, 252, 253]. The implementation of edge-based lesion segmentation often requires the use of the established edge operators such as Canny [254-256], Prewitt [257], Sobel [258], Kirsch [259] and Laplacian of Gaussian (LOG) [260]. An edge-based segmentation method based on dynamic programming using CIE L*a*b* colour space was proposed by Abbas et al. [233, 234]. However, the major challenge in the application of dynamic programming is its inability to accurately detect outline of lesion in scenarios where areas belonging to the lesion are divided into multiple tumours.

Region-based methods use a seed-based approach that groups the regions according to common image properties and relative information of the neighbouring pixels [236, 261-267]. Popular region-based methods include Fuzzy-Based Split and Merge (FBSM), J-Image Segmentation (JSEG) [262, 268], Statistical Region Merging (SRM) [269, 270], Iterative Stochastic Region Merging (ISRM) [266] and watershed [226, 271]. At this juncture, we would like to state that though there exists a similarity between edge-based lesion segmentation and region-based lesion segmentation, but both are different. Essentially, region-based segmentation methods require closed boundary to properly estimate lesion borders, whereas such requirement is not essential for edge-based segmentation. It has been argued that region-based lesion segmentation sometimes leads to over-segmentation [272]. Over-segmentation can occur when the interior of a lesion exhibits multi-coloured areas. Many advances have been recorded in the literature to resolve the aforementioned challenges, thus yielding effective region-based lesion segmentation. Ma & Tavares [273] recently proposed an algorithm built on deformable model methods to define speed function on the lightness, saturation, and colour information of a given dermoscopic image in order to estimate its lesion boundary. Geometric deformable models have been posed to implicitly represent the moving curve evolution in a way that helps to obtain desirable features (such as regions and the boundaries of the skin lesions) for shape and colour analysis simultaneously [273]. Similarly, a saliency-based segmentation method was proposed by Ahn et al. [274] via measurement of sparse reconstruction errors against image backgrounds to estimate contrast discrimination between the lesion part of a given image and the surrounding healthy skin. Saliency-based segmentation techniques help to resolve the problem of target localization, such as the difficulties in segmenting lesion image with multi-coloured objects, as well as lesion images having similar colour between the foreground and background region. The approach proposed by Olugbara et al. [275] utilized a perceptual colour difference saliency with morphological analysis to achieve a compelling segmentation result of lesions. A good future research would be to investigate how saliency segmentation can be used on lesion images with multiple saliency-regions.

In general, contour segmentation methods can be either region-focused or edge-focused. Edge related contour segmentation typically applies edge detectors to estimate stopping function for terminating contours at distinct edges, making it unusable for fuzzy edges. Region related contour segmentation computes region energy based on the mean value of lesion image intensity and consequently uses global image information to terminate contours even for indistinguishable edges. Most contour-oriented lesion segmentation techniques are more or less similar to their region-based segmentation method counterparts. The similarity is due to the usage of seed-based approach in categorising image region according to the common criteria between both methods. Contour oriented segmentation is often referred to as snakes [219]. Frequently used contour-oriented methods include adaptive snake, robust snake [276], Gradient Vector Flow (GVF) snake [277, 278], Mean-Shift based GVF [279-282], level set [263, 264, 276, 283, 284] and radial search [285, 286]. Mete & Sirakov [287] discussed enhancing active contour model with optimum parameters, including high boost filtering to achieve comparable segmentation results with other state-of-the-art segmentation methods. Similarly, Ivanovici & Stoica [288] suggested that a diffusion model for colour images can be used as external energy for active contours in order to achieve lesion segmentation by independently computing diffusion at various scales. The study reported by Yuan et al. [284] equally introduced a region-fusion-based segmentation framework by combining graph partitioning methods with chan-vese level set to achieve a comparable lesion segmentation result. In addition, Kasmi et al. [289] recently proposed a geodesic active contour (GAC) based lesion segmentation method that employs an automatic contour initialization close to the actual lesion boundary. This approach was lauded to address the sticking challenge at minimum local energy spots typically caused by noise artefacts such as hair shaft. Fig. (5) displays an example of result after applying contour-based techniques called Line Segmentation Detection (LSD) [290] on an image in order to segment occludinh hair artefacts.

The thresholding technique is adjudged to be the most adopted approach in the literature for lesion segmentation using the computation of image intensity [198, 206, 207, 221, 240, 247, 291-299]. The discussion of thresholding in this section addresses segmentation of the lesion image rather than its usage in preprocessing of images. Typically, thresholding technique involves a non-linear process of producing a binary image such as by assigning two levels to pixels below or above a specified threshold value. Thresholding can be categorised based on the parameter usage as either parametric or non-parametric. Parametric thresholding uses a set of parameters to control fitness of the model while non-parametric thresholding estimates thresholds by optimizing objective functions such as variance-based functions (cluster variance) or entropy-based functions (cross entropy). Non-parametric thresholding can be further categorised either as global thresholding based on whether thresholding is performed on an entire lesion image using a single value or as local thresholding if a lesion image is partitioned into sub images, with each image region having their respective threshold value. Most thresholding approaches discussed in the literature are seen to be global. Global thresholding can be further classified as either a point dependent, if the threshold value is determined using grey level of each pixel of the lesion image or as a region dependent if the threshold value is determined from the local property in the neighbourhood of each pixel of the lesion image. According to the literature, the implementation of a particular thresholding technique could be based on region entropy, local lesion property, histogram shape, spatiality, image attribute similarity as well as clustering [300]. Notable thresholding techniques reported in the literature for lesion image segmentation are based on popular thresholding algorithms such as Otsu [294, 301-306], type-2 fuzzy logic [297, 307], random walker [308], Kapur [304, 305, 309], Kittler [310], Ridler [311] and Sahoo [312].

In relation to image processing, clustering is sometimes referred to as a multidimensional extension of thresholding. Clustering based lesion segmentation techniques generally adopt unsupervised erudition to identify a finite set of clusters to which image pixels would be grouped. Notable clustering methods vis-à-vis skin lesion image segmentation include fuzzy c-means (FCM) [20, 238, 313, 314], k-means [193, 215, 315-317], g-means [248], density-based spatial clustering [229, 230, 318-320], grid-based spatial clustering [318], wavelet transform [313, 321, 322] and Markov random field (MRF). Recently, Khalid et al. [321] proposed an implementation of dynamic wavelet transform based on Cohen–Daubechies–Feauveau Biorthogonal to segment lesion images. The Independent Histogram Pursuit (IHP) algorithm proposed by Gómez et al. [193] revealed the possibility of segmenting lesion images using K-means clustering technique that is agnostic of colour space of the image and the number of image bands. Kockara et al. [323] used a graph clustering segmentation technique based on the soft kinetic data structure to estimate lesion border of microscopic images and consequently segmenting the lesion images. Mete el al [230]. proposed a border-driven density-based framework to identify skin lesion border by expanding regions at borders of a cluster. This approach was further improved [229] by removing preprocessing dependency. Castillejos et al. [313] proposed an ensemble of clustering based methods to segment lesion image by exploring all colour channels. Melli et al. argued in a comparison study [324] that mean shift clustering can outperform other colour clustering algorithms (median cut, k-means and fuzzy-c means) in terms of sensitivity and specificity as the number of clusters increases. Kockara et al. [325] argued that density-based clustering produces a high precision and recall rate, with low border error when used to estimate lesion image border leading to a superior result when compared to the FCM. Recently, Lemon et al. [320] advanced the usage of density clustering by proposing a skin lesion border detection method based on web computing language (WebCL) parallel density. The approach [320] takes advantage of Graphical Processing Unit (GPU) computing power of web browsers to provide quick skin lesion border detection for dermoscopic images.

The usage of morphology and statistical information, together with clustering based approaches has equally been reported in the literature. This technique involves the use of morphological features to estimate discontinuity in lesion image structures [193, 261, 326-328]. Popular morphological based clustering methods include normalized cut [328], Principal Component Analysis (PCA) [20, 57, 193, 242], linear component analysis (LDA) [329], median cut [324] and grabcut [207].

There has been a growing need to advance lesion segmentation via machine learning system. This has led to the application of several expert systems to aid segmentation of lesion images from surrounding healthy skin [238, 322, 327, 330-332]. Application of machine learning for lesion segmentation typically involves the use of expert systems to process small areas of an image for the purpose of classification. Subsequently, the network system then classifies different areas of the image based on classifications recognized by the system. Xie et al. [331] proposed a lesion segmentation algorithm for dermoscopic images by combining Self Generating Neural Network (SGNN) with Genetic Algorithm (GA). Frequently used neural network systems recorded in the literature for lesion segmentation include Radial Basis Function (RBF), Back Propagation Network (BPN), Extreme Learning Machine (ELM), Markov Random Field (MRF) [250, 327], Wavelet Network (WN), Multi-Layer Perceptron (MLP) [326] and Bayesian [327].

The literature has equally reported attempts to use ensemble of methods to improve lesion image segmentation, such as using multiple thresholding algorithms, multiple clustering approaches, region-based segmentation with neural networks or combining thresholding with region-based methods [20, 226, 240, 243, 295, 303-305, 313, 315, 331, 333-335]. In a study [48], variable threshold (based on binary imaging) and contour extraction were used to detach the shapes of the masses before determining border outline of the lesion. A study [51] utilized Laplacian filter to localize the lesion area, while zero-crossing algorithm helped the author to perform automatic outline of the lesion border. The study [41] used both pixel-based and region-based approaches to develop an algorithm, which is referred to as Dermatologists-like Tumour Area Extraction Algorithm (DTEA) to discriminate the actual tumour area from the surrounding skin. The combination of statistical clustering of the lesion colour space and hierarchical region-growing algorithm was used in a study [336] as a segmentation technique. In another study [65], segmentation was performed using a combination of bimodal histogram based on fuzzy sets region growing. Three segmentation algorithms (global thresholding, dynamic thresholding and a 3D colour clustering concept), together with fusion strategy were used [337] to obtain binary segmentation of the lesion. Pennisi et al. [272] proposed a fully automatic lesion segmentation procedure that combined edge based method (Canny) with region-based method (Delaunay triangulation) to resolve the segmentation of lesion areas.

A number of good reviews have discussed and compared notable automated lesion segmentation approaches. In the comparison study conducted by Mete & Sirakov [287], it was argued that density-based clustering performs better than Active Contour Models (ACM) when segmenting noisy lesion images. However, the ACM was adjudged to perform better when used with an optimum parameter. Celebi et al. [338] used a normalized probabilistic rand index to evaluate five different lesion segmentation approaches which include Orientation-Sensitive Fuzzy C-means Method (OSFCM) [314, 339], Dermatologist-like Tumour Extraction Algorithm (DTEA) [40, 41], mean shift clustering method [324], modified J-image Segmentation (JSEG) method [262, 268], and Statistical Region Merging (SRM) [269]. The evaluation reported the prowess of SRM as well as the consistency of DTEA across varying lesion image types. Recently, a comprehensive lesion border detection was surveyed by Celebi et al. [340] and some of the unresolved border detection issues were discussed. The latest review by Oliveira et al. [221] on computational methods for segmenting lesion images discussed several lesion boundary techniques. In the review, edge-based segmentation for lesion image was discouraged due to the fact that edge-based segmentation doesn’t consider closed boundary, and as such may produce segmented images that are not completely closed. The comparative evaluation carried out by Mendonça et al. [276] adjudged adaptive thresholding to produce the most favorable automatic segmentation of lesions, while robust snake was said to have produced a more consistent result. Silveira et al. [283] however, argued that adaptive thresholding as well as vector-valued Chan-Vese level set [341] yielded the least satisfactory result in their comparative work. In the same study [283], a proposed extension of Chan-Vese level set, called Expectation-Maximization Level Set (EM-LS) method which uses probability density functions to model lesion intensity assumptions, was observed to produce robust skin lesion segmentation result. This inconsistency in the reported evaluation could be attributed to several factors, including varying data set used as well as different comparative evaluation metrics.

It is sometimes difficult to properly analyse different automatic border detection methods for lesion images without subjective opinions resulting from the evaluation of the parameters used. Celebi et al. [338] suggested a Normalized Probabilistic Rand Index (NPRI), which takes into account the variations in the ground-truth images when evaluating different skin lesion segmentation methods. In the study [338], NPRI was adjudged to outperform the commonly used exclusive OR (XOR) measure. Garnavi et al. [342] equally proposed a weighted performance index for objective evaluation of five automated border detection methods for dermoscopy images. The weighted index was computed from six standard evaluation metric (sensitivity, specificity, accuracy, precision, border error, and similarity). The approach was further optimized in a study [343] by applying constrained non-linear multivariable optimization method in the computation of the weights.

We observed that most reported work in the literature on lesion image segmentation has been on microscopic (dermoscopic) images. From the literature, only a few efforts have been recorded in the usage of clinical (macroscopic) images for evaluating automatic lesion area segmentation approaches [249, 261, 298, 306, 335, 344]. This arguably could be attributed to the increased adoption of dermoscope in the evaluation of skin lesion images. Cavalcanti et al. [335] proposed an Independent Component Analysis (ICA) based ensemble methods to estimate lesion areas from healthy surrounding skin. In the same study [335], ensemble of thresholding and level set methods were then applied for the actual lesion boundary detection and segmentation thereof. Recently, Flores & Scharcanski [249] proposed an unsupervised dictionary learning method called Unsupervised Information Theoretic Dictionary Learning (UITDL) for estimating lesion area in macroscopic images.

The analysis made from the reports discussed in the literature suggests that a number of past works in the lesion segmentation efforts have focused on the development of algorithms based on colour information in the non-uniform space. There’s however a growing need towards optimizing segmentation algorithms in order to reduce computation time. In a bid to resolve the later, Okuboyejo et al. [207] proposed a Fast Image Segmentation (FIS) method based on the notable Contrast-Limited Adaptive Histogram Equalization (CLAHE), morphological operations, thresholding and grabcut techniques to localize lesion area from the surrounding healthy skin in a recorded time.

While most of the segmentation techniques discussed above yielded considerable promising results, the main problem with most of the approaches is that the computer-extracted regions sometimes were often smaller than the dermatologist-drawn ones (segmentation ground truth). Consequently, this makes some areas surrounding the tumour which are important features in the diagnosis to be excluded from the subsequent analysis [96, 103]. There are indications from the literature that many existing segmentation systems have high sensitivity rates towards effective diagnosis, they however experience high computing time [41, 57]. The usage of more than one algorithm for segmentation is one of the major causes of the non-realistic computing time as highlighted in a study [57]. It has also been noted that numerous past works have focused significantly on developing algorithms based on colour information in non-uniform colour spaces (disregarding the role of textural information). This has been reported to sometimes yield unsatisfactory segmentation results [234]. Another unresolved concern is the development of clinically oriented evaluation methods that can adapt variations in multiple manual borders [340]. While future research most likely would continue to use the mixture of algorithms due to increasing success rate of such approaches, more efforts should be made towards optimizing these algorithms to reduce their computing time. We would also like to suggest that the comparison of segmentation algorithms should be done on the same set of lesion images to ensure proper accuracy measure.

6.3. Feature Extraction

The primary objective of feature extraction is to quantify the macroscopic (clinical) or microscopic (dermoscopic) signs used in determining the malignancy of a skin lesion by a set of finite numerical features. Isolation of discriminating features in a given lesion image is an essential step towards effective automated lesion image classification. However, the vast variety of dermoscopic images and highly subjective definition of features characterizing these images have made the extraction of needful features a tedious task [40, 64, 96, 345, 346]. Skin distortion caused by bacterial and viral skin infections also makes analysis of features very difficult. In addition, variables such as body location, subject parameters (age), imaging parameters (lightening or camera), and direction from which lesion image is viewed and illuminated, greatly influence the resulting features that can possibly be extracted for classification purpose. These challenges typically add some overheads towards achieving automatic screening and diagnosis of medical images, especially skin lesions.

There have been numerous attempts reported in the literature to solve some of the above-mentioned challenges. One of the best approaches to address these aforementioned challenges in automating medical imaging diagnosis is to simplify the objective of the analysis and exploit some kinds of hypothetical information about the image structures. The information about the structures to be analysed can be anatomical knowledge about their typical appearance (shape, grey levels and position) or statistical knowledge of their properties (such as the greylevel of the tissues included in those structures). The images can then be classified using their morphological properties such as colour, shape, edges and texture.

The familiarity and potential discriminating power of the previously mentioned lesion diagnostic algorithm methods (such as ABCD rule of dermoscopy, ABCD criteria and Pattern analysis) have led to their usage in feature quantification. These feature descriptors can be dermoscopic, clinical or simply morphological in nature. Most reports discussed in the literature derived various discriminating lesion feature descriptors from these diagnostic algorithms, especially from the ABCD rule for dermoscopic images and ABCD criteria for clinical images. Feature descriptors used in the discrimination of skin lesion can favourably be categorised mostly as either photometric or textural. In the literature, it has been observed as common practice to use an amalgamation of various descriptors for lesion discrimination. Essentials features with corresponding discriminating properties used across the reviewed literature has been listed in Table 1.

Photometric has been seen to constitute the majority of the properties used in the literature when examining descriptors that could be used in classifying skin lesions. Photometric features include colour, island of colour, colour homogeneity & colour histogram etc. Barata et al. [64] argued that photometric features, when used with textual descriptors, yield a good result, however, the photometric features were observed to outperform the textual features if used in isolation. Stoecker et al. [347] equally suggested in his work that greater separation of melanoma from benign lesions is achieved using relative colour than using absolute colour.

Texture-based descriptors are yet another set of features that reflect the structural pattern of lesion surfaces irrespective of the colour or illumination characterizing the lesion. Texture descriptors can be categorised as spatial frequency based, statistical based, geometric based or model based. Spatial frequency based texture features are frequently associated with wavelet and ridgelet transformations. Statistical based descriptors include co-occurrence matrices and Fourier properties for describing lesion’s local neighbourhood properties. Geometric features describe skin lesion characteristics that include shape, border, symmetry, area, diameter, variance, perimeter, circularity and anisotropy. Model based textual descriptors are frequently associated with fractals and Markov random fields. Due to simplicity and ease of feature retrieval, commonly used texture descriptors include co-occurrence texture features, wavelet features and fractal-based texture features.

Co-occurrence texture descriptors such as entropy, correlation, energy, contrast, homogeneity etc. are based on co-occurrence matrices, typically the GLCM. (Fig. 9) detailed some of the frequently used texture descriptors, as well as corresponding computation. The GLCM [348] also known as greylevel spatial dependence matrix, is a form of statistical method of examining texture in relation to image pixels. GLCM outlines within a grey scale image, the probability of greylevel ἱ occurring at a distance in direction θ from grey level ĵ.

Wavelet texture features such as wavelet energy, variance and residual energy are based on wavelet transform coefficients. Fractal based texture features such as mean fractal dimension, local connected fractal dimension and global box-counting are based on fractal dimensions. One major shortfall of local fractal dimensions and local connected fractal dimensions, however, is the dependency on the choice of the maximum window size [48]. However, while it is desirable to determine features to represent these structures directly, extracting these features is often challenging primarily due to a vast variety of dermoscopy images and the highly subjective definitions of these features [40, 41, 96].

A number of different feature selection methods have been used in the literature towards ensuring appropriate discriminating features for lesion image classification. Frequent selection methods reported in the literature include Sequential Floating Forward Selection (SFFS) [337], Sequential Floating Backward Selection (SFBS) [337], Leave-One-Out, Cross Validation (xVal), Plus-I-Take-Away-r, and Genetic algorithm. Zagrouba and Barhoumi [57] argued that relative reduction of selection features could yield 50% reduction in the processing time, as well as 65% reduction in the time required to train classifiers.

While these selectors have produced positive results and contribute positively towards classification of lesion images, there resource intensive patterns are still a concern. There is thus a growing need to improve the algorithms implemented by each of the selectors to better achieve optimal feature selection process, which in turn would help reduce complexity and time-consuming computation experienced during quantification of features [349].

6.4. Classification of Lesion

Image classification involves using selected features of an image to classify pixels of the image into one of the several classes depending on specific knowledge domain. This could be in the form of training a model using a data set and then testing the model using a data set which is disjoint from the training set. Most lesion classifications are binary in nature, distinguishing between benign and malignant moles. The classification results are typically influenced by the chosen feature descriptors and strength of the classifiers. Performance of the automated classification is equally dependent on the degree of dataset population [168].

The two main classification types as reported in the literature in relation to medical imaging are supervised classification and unsupervised classification. Supervised classification uses image analysis tool to generate a statistical categorisation (such as mean and co-variance) of the reflectance of each identified information class. The completion of the categorisation then fosters effective classification by examining the reflectance of each pixel and deciding on the best matching signatures. Decision criterion such as maximum likelihood can be used for cases of overlapping signatures in order to assign pixels to the highest probable class. Unsupervised classification typically examines a large number of unknown pixels and divides them into a number of classes based on natural groupings present in the image values using procedures such as clustering. Essentially, unsupervised classification groups values that are close together in a measurement space as a single class, thus arranging the data in different classes to be comparatively separated [350].

The literature has reported the application of several classification methods for lesion images. Frequently used among these methods are the Artificial Neural Network (ANN), Decision Trees (DT), K-Nearest Neighbour (KNN), Support Vector Machine (SVM) and Regression Analysis (RA) classifiers. Similar to the neurons of a human brain, ANN comprises of an interconnected group of nodes, otherwise termed as neurons. Neural network models typically consist of both an adaptive weight that is adjusted during model training, as well as the capability to use quantitative characterization to approximate non-linear functions of their inputs.

Popular ANN methods include Back Propagation Network (BPN) [22, 40, 292, 332, 347, 351-355], Auto Associative Network (AAN) [22], Multi-Layer Perceptron (MLP) [57, 168, 224, 236, 349, 351, 355-357], and Single Layer Perceptron (SLP) [51, 358]. In the literature, extreme Learning Machine (XLM), SLP and MLP seem to be the most commonly used Feed Forward Network (FFN) methods. Main benefits of Bayesian network include its quick training capability and insensitivity to irrelevant features [298]. Sample application of ANN for lesion classification purpose can be seen in different studies [22, 40, 51, 57, 63, 168, 236, 292, 332, 352, 353, 359]. A major challenge in the application of ANN includes the excessive time that might be required in training dataset.

Bayesian network is another frequently used classifier in the space of lesion discrimination [351]. It is a probabilistic graphical model that applies Directed Acyclic Graph (DAG) to represent a set of random variables with their corresponding conditional dependencies. It should be noted that the term Bayesian network depicts the usage of Bayes rule for probabilistic inference and not necessarily implies commitment to Bayesian statistics. One major advantage in the use of Bayesian network is its insensitivity to irrelevant features. Its drawback, however, includes sometimes undesirable assumption declaring that discriminating features are independent [298]. Application and analysis of Bayesian network can be seen in different studies [224, 360-364]. In the literature, Hidden Naïve Bayes (HNB) has been observed to perform better than the Bayesian network method used [62, 361]. If a set of lesion outcomes represented by a vector v=v1, v2, …, vn is to be classified to j possible classes (Cj), then the conditional posterior probability using Bayesian rule can be expressed as:

Regression analysis is a statistical analysis for estimating the relationship between dependent (criterion) and independent variables (regression function or predictors). This typically tracks changes in the dependent variables as one of the members of the predictors is kept constant while other members of the predictors are varied. Frequently used regression analysis methods include Discriminant Function Analysis (e.g. Linear or Quadratic Regression) [40, 57, 60, 137, 253, 292, 364-367] and Logistic Regression [18, 75, 93, 102, 137, 368, 369].

Decision trees typically adopt a tree-like graph of possible decisions and the corresponding outcomes which could trigger another decision till a specific conclusion is reached [291, 361, 369, 370]. The major merit of using decision trees includes the speed at which it can be trained as well as its ease of use. Frequently used DT methods include C4.5 Decision Tree [291, 371-373], Logistic Model Tree (LMT) [291, 361, 374], Random Forest [357, 361, 370, 375], and Gradient Boosting (e.g. Adaptive Boosting: AdaBoost) [64, 291, 375-380]. Drawbacks seen in the usage of decision trees include difficulties in dealing with correlated features and the likelihood of over-fitting which typically results in excessive adjustments [298]. DT method was dubbed to perform the least in the comparative study described by Dreiseitl et al. [381], however, comparable to human expert.

The K-Nearest Neighbour (K-NN) is an algorithm that can also be applied as a classifier by storing the available cases and then classifying new cases based on the similar measurement in feature space [64, 198, 337, 344, 362]. The classifier input consists of k closest sample in the feature space, while its output result in class membership of objects being sampled. Contrary to some other classifiers, K-NN does not implement a decision boundary, however, uses the elements of the training set to estimate the density distribution of the data [381]. Hierarchical K-NN is an optimized subset of K-NN, however, it adopts both observation and feature space in its classification procedures.

The SVM is a non-probabilistic binary linear classifier that uses a learning module to analyse patterns within a collection of data for possible classification into one of the two categories. It adopts supervised learning for labelled data and an unsupervised clustering approach when data is not labelled. SVM also provides a unified framework in which different learning machine architectures can further be generated through an appropriate choice of kernel [382]. Applications of SVM can be seen in some studies [42, 130, 291, 354, 363, 369, 383-386]. In a number of studies, SVM was judged to outperform several classifiers [354, 357, 360, 363, 375]; and it is often praised for its good generalization and simplification of the non- linear data separation by means of kernel functions [298]. While the application of SVM in discriminating between melanocytic lesions has seen a number of good results, it sometimes could be very sensitive to noise hence producing a poor result. Contrary reports to the effectiveness of SVM when compared to other classifiers has equally been reported by some research works [64, 291]. SVM and MLP performed better than the counterpart classifier in the confusion matrix described in a study [357] between MLP, K-NN, Random Forest (RF) and SVM. In a similar study described by Dreiseitl et al. [381], logistic Regression, ANN and SVM produced good discriminating results for PSL compared to KNN and Decision Tree methods.

Kreutz et al. [336] argued for the need to incorporate a combination of expert systems in classifying lesion images to enable data set to be split into regions where each expert system works effectively. Results from each expert system can then be aggregated by a gating network. This is to help resolve recurrent challenges faced when training a single expert system to classify varying degrees of input space. In effect, when input space is separated and targeted, scalability and interpretability of solutions increase. Similarly, A Multiple Expert-Based Melanoma Recognition System for Dermoscopic Images of Pigmented Skin Lesions has been proposed by Rahman and Bhattacharya [198, 299] by using combination rules generated with the application of Bayes’ theorem to produce a probabilistic output. The comparative study discussed by Ruiz et al. [351] equally argued that collaborative classifiers produced better classification compared to the usage of individual classifiers.

Furthermore, Dreiseitl et al. [381] suggested that linear factors contribute to a better discrimination compared to non-linear elements in the classifying models. This was proved in the comparative analysis [381] between K-NN, Logistic Regression, ANN, DT and SVM, where linear method (logistic regression) outperformed non-linear counterpart. Other remarkable classifiers as reported in the literature include Lacunarity analysis [96] and Markov Random Field MRF [246]. The literature also records the use of rule-based process for classifying skin lesion images. Frequently used rule-based procedures include Pattern Analysis, ABCD rule, ELM 7-point checklists, Menzies score, 7 Features for melanoma. Notable results have been recorded in the literature by various classification methods; however, there still exists some unresolved concerns in relation to effective lesion classification. Highlights of the issues include the great unbalance between lesion image classes, the difficulty in defining discriminating visual features and the effect of multiplicities of some lesion image classes. The execution speeds of the classification algorithms and resource intensive nature of some of these classifiers have posed a need for a more optimized approach, especially when considering mobile portability of these solutions.