**16-825 : Point Cloud Segmentation and Classification**
Student name: Abhishek Pavani
(#) 1. Classification Model
In this part, I implemented pointNet architecture for Classification. This is one of the most popular architectures in the field of point cloud processing. I used the same architecture as mentioned in the paper [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://arxiv.org/pdf/1612.00593.pdf).
**Test accuracy: 97.80%**
(##) Output
(###) Correct Predictions
|Category| Point cloud | Ground Truth Class | Predicted Class |
|--------|-----------------------|-----------------------|-----|
|Chair| |0|0|
|Chair| |0|0|
|Chair| |0|0|
|Vase| |1|1|
|Vase| |1|1|
|Vase| |1|1|
|Lamp| |2|2|
|Lamp| |2|2|
|Lamp| |2|2|
(###) Incorrect Predictions
| Point cloud | Ground Truth Class | Predicted Class |
|-----------------------|-----------------------|-----|
| |0|1|
| |1|2|
| |2|1|
| |1|2|
| |2|1|
| |2|1|
| |2|1|
| |2|0|
| |2|0|
(#) 2. Point Cloud Segmentation
In this part, I used a similar architecture as I used in case of classification, but just replaced the classification head with the segmentation head as proposed in the paper [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://arxiv.org/pdf/1612.00593.pdf).
**Test Accuracy: 90.30%**
(##) Output
(###) Bad Predictions
| Ground Truth Point cloud |Predicted Point Cloud|
|---------------------------|---------------------|
| ||
| ||
| ||
| ||
| ||
(###) Good Predictions
| Ground Truth Point cloud |Predicted Point Cloud|
|---------------------------|---------------------|
| ||
| ||
| ||
| ||
| ||
(#) 3. Robustness Analysis
In this section I check how robust my models are to rotations and sparse sampling. This is important because we might not always have oriented or upright point cloud and might need a model that is robust to these transformations.
(##) Rotation
In this section, I check how my model performs on rotated point clouds. I rotate the point cloud by different angles and results for the same have been shown below.
To perform the rotations, I chose random rotation values and the created generated euler angles (ZYX order) and created a rotation matrix out of the same
I then multiplied this matrix with the test_data point cloud to generate the rotated point cloud.
I think model has a hard time making predictions when the rotations are too steep. This is mainly because we did not perform any form of data augmentation when we trained the model. So the model learns only upright configurations of the point clouds.
(##) Rotation along the Z axis
| Angle 30 | Angle 45| Angle 60| Angle 90| Angle 120|
|----------|-----------|----------|------|----------|
||||||
||||||
|87.35%|69.77%|55.4%|31.3%|20.8%|
(##) Rotation for segmentation along the Z axis
| Angle 45| Angle 90| Angle 120| Angle 180|
|----------|-----------|----------|------|----------|
|||||
|58.55%|31.33%|35.77%|35.76%|
(##) Rotation along all 3 axis by the same angle (ZYX rotation))
| Angle 30 | Angle 45| Angle 60| Angle 90| Angle 120|
|----------|-----------|----------|------|----------|
||||||
||||||
|57.08%| 32.63%| 42.6%| 43.33%| 26.58%|
(##) Sparse points
Inorder to reduce the sparsity, I just sampled less points per obeject and tried evaluating the model for classification task. The results are shown below.
It can be seen that the model is very robust to sparse points. The accuracy of the model is 25% for 10 points and 79% for 50 points. I think one of the reason the accuracy is still that high even with only 10 points is because,
there are just 3 classes that the model can predict from and there is a 33% chance of picking up any random class and assigning it the point cloud. If there were more classes, I think it should have failed to predict the class correctly and the accuracy would have dropped further.
| Num points| Predicted Point Cloud| Accuracy %|
|-----------|---------------------|----------|
|10000|| 98.5|
|1000|| 97.16|
|100|| 93.39|
|50|| 79.46|
|10|| 25.07|
| Num points| Predicted Point Cloud| Accuracy %|
|-----------|---------------------|----------|
|10000|| 90.3|
|1000|| 88.66|
|100|| 83.18|
|10|| 66.08|
(##) 4. Bonus Question - Exploring other architectures
(#) 1. Classification Model
In this part, I implemented pointNet architecture for Classification. This is one of the most popular architectures in the field of point cloud processing. I used the same architecture as mentioned in the paper [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://arxiv.org/pdf/1612.00593.pdf).
**Test accuracy: 97.80%**
(##) Output
(###) Correct Predictions
|Category| Point cloud | Ground Truth Class | Predicted Class |
|--------|-----------------------|-----------------------|-----|
|Chair|
|0|0|
|Chair|
|0|0|
|Chair|
|0|0|
|Vase|
|1|1|
|Vase|
|1|1|
|Vase|
|1|1|
|Lamp|
|2|2|
|Lamp|
|2|2|
|Lamp|
|2|2|
(###) Incorrect Predictions
| Point cloud | Ground Truth Class | Predicted Class |
|-----------------------|-----------------------|-----|
|
|0|1|
|
|1|2|
|
|2|1|
|
|1|2|
|
|2|1|
|
|2|1|
|
|2|1|
|
|2|0|
|
|2|0|
(#) 2. Point Cloud Segmentation
In this part, I used a similar architecture as I used in case of classification, but just replaced the classification head with the segmentation head as proposed in the paper [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://arxiv.org/pdf/1612.00593.pdf).
**Test Accuracy: 90.30%**
(##) Output
(###) Bad Predictions
| Ground Truth Point cloud |Predicted Point Cloud|
|---------------------------|---------------------|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(###) Good Predictions
| Ground Truth Point cloud |Predicted Point Cloud|
|---------------------------|---------------------|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(#) 3. Robustness Analysis
In this section I check how robust my models are to rotations and sparse sampling. This is important because we might not always have oriented or upright point cloud and might need a model that is robust to these transformations.
(##) Rotation
In this section, I check how my model performs on rotated point clouds. I rotate the point cloud by different angles and results for the same have been shown below.
To perform the rotations, I chose random rotation values and the created generated euler angles (ZYX order) and created a rotation matrix out of the same
I then multiplied this matrix with the test_data point cloud to generate the rotated point cloud.
I think model has a hard time making predictions when the rotations are too steep. This is mainly because we did not perform any form of data augmentation when we trained the model. So the model learns only upright configurations of the point clouds.
(##) Rotation along the Z axis
| Angle 30 | Angle 45| Angle 60| Angle 90| Angle 120|
|----------|-----------|----------|------|----------|
|
|
|
|
|
|
|
|
|
|
|
|
|87.35%|69.77%|55.4%|31.3%|20.8%|
(##) Rotation for segmentation along the Z axis
| Angle 45| Angle 90| Angle 120| Angle 180|
|----------|-----------|----------|------|----------|
|
|
|
|
|
|58.55%|31.33%|35.77%|35.76%|
(##) Rotation along all 3 axis by the same angle (ZYX rotation))
| Angle 30 | Angle 45| Angle 60| Angle 90| Angle 120|
|----------|-----------|----------|------|----------|
|
|
|
|
|
|
|
|
|
|
|
|
|57.08%| 32.63%| 42.6%| 43.33%| 26.58%|
(##) Sparse points
Inorder to reduce the sparsity, I just sampled less points per obeject and tried evaluating the model for classification task. The results are shown below.
It can be seen that the model is very robust to sparse points. The accuracy of the model is 25% for 10 points and 79% for 50 points. I think one of the reason the accuracy is still that high even with only 10 points is because,
there are just 3 classes that the model can predict from and there is a 33% chance of picking up any random class and assigning it the point cloud. If there were more classes, I think it should have failed to predict the class correctly and the accuracy would have dropped further.
| Num points| Predicted Point Cloud| Accuracy %|
|-----------|---------------------|----------|
|10000|
| 98.5|
|1000|
| 97.16|
|100|
| 93.39|
|50|
| 79.46|
|10|
| 25.07|
| Num points| Predicted Point Cloud| Accuracy %|
|-----------|---------------------|----------|
|10000|
| 90.3|
|1000|
| 88.66|
|100|
| 83.18|
|10|
| 66.08|
(##) 4. Bonus Question - Exploring other architectures