If you are new to setting up a mail server be cautious that you do not open your server accidently and relay mail...
One very simple way of testing it is to run:
$ telnet relay-test.mail-abuse.org
This will attempt to connect back to your machine and run a series of mail relaying tests against it. The success or failure will be printed at the end.
You may need to turn off firewall temporarily during the test or allow for outgoing connection through the firewall.
EmailSecurityGrader is a another useful too for Open Relay
You can also run a complete security test on http://www.EmailSecurityGrader.com – it has an extensive Open Relay test (including % hacks) and also includes several other email security tests (SPF, DNSBL/Spam Blacklist) which noawadays are at least as important as Open Relay.
rt.njabl.org for telnet mail relay testing
Alternately, If you would like your server tested, use telnet to connect to port 2500 on rt.njabl.org from the server you want tested. Your server will be tested and you will see the results of the test as it is run.
Note: If you are not sure how your system was used as an open relay, you can telnet as instructed above and the SMTP conversation will display in real time as your system is (re)tested, demonstrating the combination of to/from addresses which result in your system acting as an open relay.
$ telnet rt.njabl.org 2500