Predicting Boston House Prices with Keras in Python

The Boston Housing dataset is one of the most well-known datasets in the machine learning community. It contains information collected by the U.S. Census Service concerning housing in the area of Boston, Massachusetts. The goal is to predict the median value of owner-occupied homes (in $1000s) based on various features such as the crime rate, average number of rooms per dwelling, and more. In this blog, we'll walk through how to build a neural network model using Keras in Python to predict housing prices.

Predicting Boston House Prices with Keras

Predicting Boston housing prices using Keras and TensorFlow involves leveraging deep learning techniques to estimate the median value of homes based on various factors such as crime rate, average number of rooms, and accessibility to highways. Keras, with TensorFlow as its backend, provides an intuitive and flexible way to build, train, and evaluate neural networks. By normalizing the input data and constructing a model with multiple layers, the network learns complex patterns and relationships within the dataset. The process includes splitting the data into training and testing sets, optimizing the model using techniques like cross-validation, and fine-tuning parameters to minimize prediction errors. This approach demonstrates how modern deep learning frameworks can be effectively applied to traditional regression problems, offering insights into real-world applications of AI in predicting economic and social trends.

Boston House Prices Dataset with Keras in Python

The Boston Housing Dataset is a widely used dataset in machine learning for regression tasks and is particularly popular for demonstrating predictive modeling techniques. It contains 506 samples and 13 numerical features, including per capita crime rate, proportion of non-retail business acres, nitric oxide concentration, and average number of rooms per dwelling, among others. The target variable is the median value of owner-occupied homes in thousands of dollars. In TensorFlow, this dataset serves as an excellent example for building and training models to predict continuous outcomes. TensorFlow provides built-in functions to load and preprocess this dataset, enabling users to experiment with various neural network architectures and optimization techniques to improve prediction accuracy. The Boston Housing dataset consists of 506 samples and 13 feature variables, which are:

1. CRIM - Per capita crime rate by town

2. ZN - Proportion of residential land zoned for lots over 25,000 sq. ft.

3. INDUS - Proportion of non-retail business acres per town

4. CHAS - Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)

5. NOX - Nitric oxides concentration (parts per 10 million)

6. RM - Average number of rooms per dwelling

7. AGE - Proportion of owner-occupied units built prior to 1940

8. DIS - Weighted distances to five Boston employment centers

9. RAD - Index of accessibility to radial highways

10. TAX - Full-value property tax rate per $10,000

11. PTRATIO - Pupil-teacher ratio by town

12. B - 1000(Bk−0.63)21000(Bk−0.63)2 where BkBk is the proportion of Black individuals by town

13. LSTAT - Percentage of lower status of the population

The target variable is MEDV - Median value of owner-occupied homes in $1000s.

Implementing Neural Network on Boston House Prices Dataset with Keras in Python

Implementing a neural network on the Boston Housing dataset using Keras in Python involves several key steps to predict housing prices based on various features like crime rate, number of rooms, and tax rates. First, the dataset is loaded and preprocessed, where the features are normalized to improve the model's performance. A neural network is then constructed with Keras, typically using a Sequential model with multiple dense layers—each with ReLU activation functions. The model is trained using the Mean Squared Error (MSE) as the loss function and Mean Absolute Error (MAE) as a metric for easier interpretation of results. After training, the model's performance is evaluated on test data, providing insights into its predictive accuracy. This approach showcases how deep learning techniques can be effectively applied to regression problems in real-world datasets.

Step 1: Installing all Necessary Libraries

To install Keras, NumPy, and Matplotlib, you can use pip, the package installer for Python. Below are the commands to install each of these libraries:

• Keras: Keras is included as part of TensorFlow, so installing TensorFlow will also install Keras.

pip install tensorflow

• NumPy: NumPy is a fundamental package for scientific computing in Python.

pip install numpy

• Matplotlib: Matplotlib is a plotting library used for creating static, animated, and interactive visualizations in Python.

pip install matplotlib

If you want to install all three packages at once, you can use a single command:

pip install tensorflow numpy matplotlib

This will install the latest versions of TensorFlow (which includes Keras), NumPy, and Matplotlib. After installation, you can start using these libraries in your Python projects.

Step 2: Loading Boston Housing Dataset

Keras provides a direct way to load the Boston Housing dataset. Here’s how you can load it:

from keras.datasets import boston_housing

# Load the data

(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()

print(f'Training data shape: {train_data.shape}')

print(f'Testing data shape: {test_data.shape}')

Output for the code above:

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/boston_housing.npz

57026/57026 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step

Training data shape: (404, 13)
Testing data shape: (102, 13)

Step 3: Preprocessing Boston Housing Dataset

Before feeding the data into the model, it’s crucial to preprocess it. Neural networks perform better when the input data is normalized.

# Normalize the data

mean = train_data.mean(axis=0)

std = train_data.std(axis=0)

train_data = (train_data - mean) / std

test_data = (test_data - mean) / std

Step 4: Building Neural Network in Keras

We’ll build a simple neural network with two hidden layers. Keras makes it straightforward to define a Sequential model:

from keras import models

from keras import layers

# Build the model

def  build_model():

     model = models.Sequential()

     model.add(layers.Dense(64, activation='relu', input_shape=(train_data.shape[1],)))

     model.add(layers.Dense(64, activation='relu'))

     model.add(layers.Dense(1)) # No activation function as it's a regression problem

     model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])

     return model

model = build_model()

• 64 neurons with ReLU activation in each hidden layer.

• 1 neuron in the output layer since we're predicting a single continuous value.

• MSE (Mean Squared Error) is used as the loss function.

• MAE (Mean Absolute Error) is tracked as a metric for easier interpretation.

Step 5: Training Neural Network in keras

We can now train the model using the training data. Since the dataset is small, we’ll use k-fold cross-validation to ensure that the model generalizes well.

import numpy as np

# K-fold cross-validation

k = 4

num_val_samples = len(train_data) // k

num_epochs = 100

all_mae_histories = []

for i in range(k):

     print(f'Processing fold #{i}')

     val_data = train_data[i num_val_samples: (i + 1) num_val_samples]

     val_targets = train_targets[i num_val_samples: (i + 1) num_val_samples]

     partial_train_data = np.concatenate([train_data[:i num_val_samples], train_data[(i + 1) num_val_samples:]], axis=0)

    partial_train_targets = np.concatenate([train_targets[:i num_val_samples], train_targets[(i + 1) num_val_samples:]], axis=0)

    model = build_model()

     history = model.fit(partial_train_data, partial_train_targets, epochs=num_epochs, batch_size=1, verbose=0, validation_data=(val_data, val_targets))

     mae_history = history.history['val_mae']

    all_mae_histories.append(mae_history)

# Average MAE across all folds

average_mae_history = [np.mean([x[i] for x in all_mae_histories]) for i in   range(num_epochs)]

Output for the code above:

Processing fold #0
Processing fold #1
Processing fold #2
Processing fold #3

Step 6: Evaluating Neural Network in keras

After the model is trained, it's time to evaluate it on the test data to see how well it performs:

# Build the final model

model = build_model()

model.fit(train_data, train_targets, epochs=80, batch_size=16, verbose=0)

# Evaluate the model on the test data

test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)

print(f'Test Mean Squared Error: {test_mse_score}')

print(f'Test Mean Absolute Error: {test_mae_score}')

Output for the code above:

4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - loss: 12.5410 - mae: 2.4532 
Test Mean Squared Error: 17.762845993041992
Test Mean Absolute Error: 2.6534388065338135

Step 7: Visualizing the Results using matplotlib

Visualizing the training and validation loss can give us insights into how well the model performs and whether it's overfitting.

import matplotlib.pyplot as plt

plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)

plt.xlabel('Epochs')

plt.ylabel('Validation MAE')

plt.show()

Output for the code above:

Full code for Implementing Neural Network on Boston Housing Dataset with Keras in Python

The full code for implementing a neural network on the Boston Housing dataset using Keras in Python includes loading and normalizing the data, building a Sequential model with dense layers, training the model using Mean Squared Error, and evaluating its performance on test data. The code also includes visualization of training progress with Matplotlib, providing a comprehensive guide to developing and analyzing deep learning models for regression tasks.

from keras.datasets import boston_housing

from keras import models

from keras import layers

import numpy as np

import matplotlib.pyplot as plt

# Load the data

(train_data, train_targets), (test_data, test_targets) = boston_housing.load_data()

print(f'Training data shape: {train_data.shape}')

print(f'Testing data shape: {test_data.shape}')

# Normalize the data

mean = train_data.mean(axis=0)

std = train_data.std(axis=0)

train_data = (train_data - mean) / std

test_data = (test_data - mean) / std

# Build the model

def  build_model():

    model = models.Sequential()

     model.add(layers.Dense(64, activation='relu', input_shape=(train_data.shape[1],)))

     model.add(layers.Dense(64, activation='relu'))

     model.add(layers.Dense(1)) # No activation function as it's a regression problem

     model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])

     return model

# K-fold cross-validation

k = 4

num_val_samples = len(train_data) // k

num_epochs = 100

all_mae_histories = []

for i in range(k):

     print(f'Processing fold #{i}')

     val_data = train_data[i  num_val_samples: (i + 1)  num_val_samples]

     val_targets = train_targets[i  num_val_samples: (i + 1)  num_val_samples]

     partial_train_data = np.concatenate([train_data[:i  num_val_samples], train_data[(i + 1)  num_val_samples:]], axis=0)

    partial_train_targets = np.concatenate([train_targets[:i  num_val_samples], train_targets[(i + 1)  num_val_samples:]], axis=0)

    model = build_model()

     history = model.fit(partial_train_data, partial_train_targets, epochs=num_epochs, batch_size=1, verbose=0, validation_data=(val_data, val_targets))

     mae_history = history.history['val_mae']

    all_mae_histories.append(mae_history)

# Average MAE across all folds

average_mae_history = [np.mean([x[i] for x in all_mae_histories]) for i in   range(num_epochs)]

# Build the final model

model = build_model()

model.fit(train_data, train_targets, epochs=80, batch_size=16, verbose=0)

# Evaluate the model on the test data

test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)

print(f'Test Mean Squared Error: {test_mse_score}')

print(f'Test Mean Absolute Error: {test_mae_score}')

plt.plot(range(1, len(average_mae_history) + 1), average_mae_history)

plt.xlabel('Epochs')

plt.ylabel('Validation MAE')

plt.show()

Conclusion

In conclusion, the Boston Housing dataset provides a practical example of how neural networks can be applied to real-world regression problems using Keras in Python. By following a systematic approach—beginning with data preprocessing, followed by model building, training, evaluation, and finally visualization—we can leverage the power of deep learning to make accurate predictions. Installing essential libraries like Keras (via TensorFlow), NumPy, and Matplotlib ensures a smooth setup for developing and experimenting with neural networks. Through this exercise, we gain valuable insights into the versatility and effectiveness of deep learning models in handling complex datasets, paving the way for more advanced applications in various domains.