Osteoporosis is a worldwide problem associated with an increasing number of fragility fractures. Currently, the standard for identifying patients at risk of fragility fracture is through Dual X-ray Absorptiometry (DXA). Different altenatives have been proposed, such as magnetic resonance imaging (MRI), three-dimensional X-rays, ultrasound or algorithms providing scores from clinical data. Among ultrasonic techniques, Bi-Directional Axial Transmission (BDAT) has been used to classify patients with or without fragility fractures, initially using”classical” ultrasound parameters, such as velocities and latter using Support Vector Machine and automatic features, with performances close to the gold standard DXA. The aim of this study was to investigate the use of Convolutional Neural Networks (CNN) applied to patient classification using ultrasonic guided wave spectrum images, using a previous database of post menopausal women with or without fragility fractures. Two networks will be tested, a reference one, ResNet, successfully applied in classification and diagnosis in medical images, and a tailored one, denoted BDAT-Net, which hyperparameters will be optimized through a grid approach. The obtained accuracy, using BDAT-Net and clinical data (age, body mass index, cortisone intake) was found equal to 0.66 [0.64-0.69] comparable with the one obtained with DXA and significantly better than the one obtained with ResNet. These encouraging results open the door to the use of robust ultrasonic devices for fracture risk assessment, in particular in countries where DXA is not widely available.