After adding a port forwarding rule, and making sure it worked, I decided to remove to rule.

Deleting the port forwarding configuration doesn't stop the rule from working (i.e. the port still gets forwarded).

Adding a new rule overrides the first one, but removing the new rule reverts to the first port forwarding configuration, even though there's no rule listed.

Steps to reproduce:

1. Make sure there are no port forwards configured;

2. Add a port forward PORT_X to HOST_A;

3. Test the forwarding;

4. Remove the rule;

5. Test again, the rule still applies;

6. Add a forwarding rule on the same PORT_X to HOST_B;

7. Test and see that HOST_B is receiving the connection;

8. Remove the rule;

9. Test and see that HOST_A is again receiving the connection, even though there's no active rule.

Workaround: rebooting the system removes the rule permanently.