For a tutorial, we're going to write a program I call a port forwarder. Our port forwarder will accept incoming connections on one TCP/IP port (port 1), or socket, and makes matching outgoing connections to a different port (port 2). It then sends any data sent to port 1 to port 2, and any data sent to port 2 to port 1. The net effect is a program that seems to make one resource (like www.microsoft.com port 80) to also appear at another place (like fred.omnifarious.org port 3000).
I use the program we will write to make specific tunnels around firewalls. I also use it to send IRC connections through an encrypted IP tunnel to my computer at home, then forward them to their eventual destination.
Here's a diagram (it may not look right in browsers other than Mozilla, which is the only browser I've found that correctly handles alpha channels) of a typical running PortForward session:
As you can see, anything I type into my xchat client goes to the port forwarder program, then is sent from there to an openprojects.net IRC server. Any data coming from the openprojects.net IRC server gets sent to the port forwarder program, and it sends it to my xchat client. If, for example, I cannot get directly to the irc.openprojects.net server from someplace, I can install the port forwarder someplace I can get to, and it will act as a sort of proxy and enable me to get there.
I have two other requirements of the port forwarder that make it hard to implement in a traditional way. One requirement is that the forwarder be able to forward multiple connections at the same time. The other is that I allow several different ports to be forwarded with the same executable.
For an example of the first requirement, I may want to allow many people to get to irc.openprojects.net from inside my network. Or I may be forwarding an HTTP connection, and therefor need to allow several simultaneous connections.
For the second requirement, it may be that the firewall blocks all connections to port 6667 (the IRC port) and I want to connect to two different IRC servers at the same time. Or maybe, I need to do both ssh and imaps to my computer at home.
This project is small enough that the design can be created along with the code, so I won't have a separate coding step.
Looking at the requirements, the configuration information needed to implement them is pretty cut and dried. We need a list consisting of (port to listen on, address to connect to, port to connect to). These things would probably best go in a config file, but for simplicity's sake, I'm going to just read them from stdin. Here's a code snippet that will do just that.
unsigned int inport = 0; string outaddr; unsigned int outport = 0; while (cin) { cin >> inport >> outaddr >> outport; cout << "Accepting connections on port " << inport << " and redirecting data to a connection to " << outaddr << " port " << outport << "\n"; }
Of course, this code bucks Unix tradition, and actually prints out what it read in, even if it wasn't in error, but no matter. As you can see, this just reads in the config file, and does nothing about acting on it. We'll get to some of that in a bit.
Next thing we need is a way to actually listen on the ports we're supposed to listen on. Looking through the StreamModule library, it looks like SockListenModule from StrMod/SockListenModule.h is what will fit that need.
The SockListenModuleModule constructor looks like this:
SockListenModule(const SocketAddress &bind_addr, unievent::Dispatcher &disp, unievent::UnixEventRegistry &ureg, int qlen = 1);
It takes some mysterious arguments. The first, SocketAddress
&bind_addr
, is the address to listen on, the address the listening
socket is 'bound' to. One might think a port number is enough, but sometimes
you want to listen at a particular IP address (the computer may have several)
so that only certain things can connect to you.
The second is a Dispacher. The Dispatcher is used to to queue up events for dispatching. We should worry about the details of this now, except to not that one is needed.
The third is a UnixEventRegistry. A UnixEventRegistry is used to generate StreamModule system events based on operating system events. In this case, Unix telling us that a file descriptor we're interested in can now be written to or read from without blocking our program. The SockListenModule needs to know this for two reasons. One reason is that it needs to tell the SocketModule created when it processes an incoming connection. The second reason is that the OS tells it that there is an incoming connection to accept, by saying the file descriptor can be read without blocking.
The last, int qlen
is given directly to the operating system
as a parameter when creating the listening socket. It tells the operating
system how many connections it can queue up without the program accepting
any. Too low, and people will get 'connection refused' errors, even though
our program is running because it hasn't processed an incoming connection.
Too high, and we'll get a huge backlog of incoming connections to work
through. Since we don't expect a very high volume of incoming connections
(presumably PortForward will be used on a personal basis, and not as a
frontend for a popular web or IRC server), we will set it at a fairly
conservative '2'.
Now, of these parameters, the Dispatcher, UnixEventRegistry, and SocketAddress are abstract base classes. In order to instantiate one of them, we have to figure out which derived class we need.
In the case of SocketAddress, it's easy. There is currently only one
derived class, and moreover, it make perfect sense given what it's supposed
to represet. That class is InetAddress
. SocketAddress represents
any address a socket could connect or bind to. These would include IPX
addresses, Appletalk addresses, IP addresses and so on. InetAddress
represents an IP address. Since the port forwader is for TCP/IP connections,
it's the kind of address we need.
InetAddress has several constructors. We will pick the one that best represents that data we have, the port to listen on:
InetAddress(U2Byte port);
For Dispatcher, we will use SimpleDispatcher
because we don't
need any more complicated facilities that may be provided by a more
sophisticated dispatcher. And, in truth, there is no more sophisticated
dispatcher written yet.
SimpleDispatcher only has one constructor, and that takes no arguments:
SimpleDispatcher();
For UnixEventRegistry, there is also only the UnixEventPoll derived class. There will be other UnixEventRegistry implementations that use select on machines that don't have poll, and also SIGIO or the BSD kernel queue on some Unices. For now though, there is only UnixEventPoll.
UnixEventPoll also only has one constructor. That constructor takes a Dispatcher as an argument, meaning that the Dispatcher will have to be created before the UnixEventPoll:
UnixEventPoll(Dispatcher *disp);
Then, there's a class to tie them together. This class exists because, in the future, the Dispatcher interface is going to be greatly simplified, and UnixEventPoll won't be able to count on a certain technique to get itself run to look for other Unix events when the queue is empty, or when the system is busy.
So, there's a glue class, RegistryDispatcherGlue, that ties together the current Dispatcher interface, and a UnixEventRegistry implementation (like UnixEventPoll is). This glue class registers the appropriate events with the Dispatcher and calls UnixEventRegitry's doPoll method at the appropriate times, with the appropriate values for 'wait'.
Putting this all together yields code that looks something like this:
SimpleDispatcher dispatcher; UnixEventPoll pollmanager(&dispatcher); RegistryDispatcherGlue glue(&dispatcher, &pollmanager); do { U2Byte inport = 0; string outaddr; unsigned int outport = 0; cin >> inport >> outaddr >> outport; if (cin) { InetAddress listenaddr(inport); SockListenModule *slm = new SockListenModule(listenaddr, dispatcher, pollmanager, 2); cout << "Accepting connections on port " << inport << " and redirecting data to a connection to " << outaddr << " port " << outport << "\n"; } } while (cin);
This, of course, isn't at all a complete program. Aside from the lack of
any function (not even main
) it has a big memory leak, and will
also implicitly (because of the memory leak) leave dangling pointers to
dispatcher and pollmanager. It doesn't even really do anything useful,
though, if you (under Linux) enclose it in a main, put a long sleep at the
end, run it, giving a file of port mappings on stdin, then look at the output
of netstat -aA inet
, you'll see that all the ports are being
listened to, so it does do a little something.
The main things missing if we are to implementing our requirements are these: First, we aren't tracking the place to connect to if we should get an incoming connection on any of our listening ports. Second, though we are putting out listening sockets, we aren't doing anything with any incoming connections. Lastly, we aren't making any outgoing connections in response to our incoming ones.
Lets tackle the first thing. Obviously, we need a data structure that tracks the listening socket, and the outgoing address. It also might be nice to remember what the port number we are listening to. Since we'll need these values in the form of SocketAddresses when we use them, we might as well store them as SocketAddresses.
class Listener { public: // Lets alias a few types that we use all over in this class to make things // easier on our fingers. typedef ::strmod::ehnet::SocketAddress SocketAddress; typedef ::strmod::unievent::Dispatcher Dispatcher; typedef ::strmod::unievent::UnixEventRegistry UnixEventRegistry; Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg); ~Listener(); private: SockListenModule *listener_; SocketAddress * const from_; SocketAddress * const to_; }; Listener::Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg) : listener_(NULL), from_(from.Copy()), to_(to.Copy()) { listener_ = new SockListenModule(*from_, disp, ureg, 2); } Listener::~Listener() { delete listener_; delete from_; delete to_; }
That was pretty straightforward. One thing that is a bit tricky is the
from.Copy()
and to.Copy()
bits in the
constructor.
As I've said before, a SocketAddress is an abstract base class, so you can't actually have a SocketAddress. Also, the SocketAddress objects we are being passed are not 'owned' by the Listener, so it needs copies that it owns so it can control their lifetime. This also means that they need to be destroyed in the destructor.
Hence, the Copy()
method. That method is defined as a virtual
function on SocketAddress, and its intent is to return a full copy of the
object. If the SocketAddress &
refers to an InetAddress, it
returns a copy of the InetAddress. If it refers to a UnixSocketAddress, it
returns a UnixSocketAddress and so on.
Now, to put it all together. Since we want to create a program here that you could compile, this program will contain the proper #include and namespace directives.
#include <StrMod/SockListenModule.h> #include <EHnet++/SocketAddress.h> #include <EHnet++/InetAddress.h> #include <UniEvent/Dispatcher.h> #include <UniEvent/SimpleDispatcher.h> #include <UniEvent/UnixEventRegistry.h> #include <UniEvent/UnixEventPoll.h> #include <UniEvent/RegistryDispatcherGlue.h> #include <iostream> #include <string> using strmod::strmod::SockListenModule; using strmod::unievent::Dispatcher; using strmod::unievent::UnixEventRegistry; // The EHnet++ library has not yet been converted to use namespaces. class Listener { public: // Lets alias a few types that we use all over in this class to make things // easier on our fingers. typedef ::strmod::ehnet::SocketAddress SocketAddress; typedef ::strmod::unievent::Dispatcher Dispatcher; typedef ::strmod::unievent::UnixEventRegistry UnixEventRegistry; Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg); ~Listener(); private: SockListenModule *listener_; SocketAddress * const from_; SocketAddress * const to_; }; Listener::Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg) : listener_(NULL), from_(from.Copy()), to_(to.Copy()) { listener_ = new SockListenModule(*from_, disp, ureg, 2); } Listener::~Listener() { delete listener_; delete from_; delete to_; } int main() { using ::strmod::lcore::U2Byte; using ::strmod::ehnet::InetAddress; strmod::unievent::SimpleDispatcher dispatcher; strmod::unievent::UnixEventPoll pollmanager(&dispatcher); strmod::unievent::RegistryDispatcherGlue glue(&dispatcher, &pollmanager); using std::cin; using std::cout; using std::string; do { U2Byte inport = 0; string outaddr; unsigned int outport = 0; cin >> inport >> outaddr >> outport; if (cin) { InetAddress listenaddr(inport); InetAddress destaddr(outaddr, outport); Listener *listener = new Listener(listenaddr, destaddr, dispatcher, pollmanager); cout << "Accepting connections on port " << inport << " and redirecting data to a connection to " << outaddr << " port " << outport << "\n"; } } while (cin); return 0; }
That code should compile if you have the libNet library from StreamModule installed somewhere with its include files. It still has a nasty memory leak, though it keeps track of the port to connect to. Lets fix the leak by adding the following lines in the appriate places:
#include <list> typedef std::list<Listener *> ListenerList; ListenerList listeners; listeners.push_front(listener); // Reverse order so they get deleted in reverse order later. for (ListenerList::iterator i = listeners.begin(); i != listeners.end(); ++i) { delete *i; *i = 0; }
I bet you can figure out where to add those. If you want to see if you're right, see the next place where the code is written out in full again.
This fixes the problem of keeping track of where to connect. The next task is to do something with incoming connections. This gets a bit more complicated and involves more of the framework.
A Dispatcher is supposed to provide an implementation of an Event queue. Things can add (post) events to the queue, and the main program can tell the Dispatcher to trigger them. This is used for couple of purposes.
One purpose is for callbacks. Oftentimes, there is a need for things of a lower level of abstraction (like buttons on a GUI) to call back to things at a higher level to tell them something important has happened.
Another purpose is for cooperative multitasking. Things can post an event when they are only partly done with a task, and that event will be fired so it can complete its task sometime later, after some other things have had a chance to run. Since control is voluntarily surrendered, no context information needs to be explicitly kept by the event mechanism. Also control flow doesn't involve any interrupts, and is likely to not have an adverse effect on CPU caches and such.
In our case, the Dispatcher will mainly be used for callbacks, though we won't yet deal directly with events.
One of the more important generators of callback events is the UnixEventRegistry. It's job is to get the OS to tell it when some event (like a file descriptor becoming readable or writeable) has happened. It then posts this event to the Dispatcher. That's why the UnixEventRegistry needs a Dispatcher as a constructor argument, and also why things that work with the OS (like SockListenModule) need to know about a UnixEventRegistry.
SockListenModule opens up a listening socket in its constructor, and registers an interest in whether or not it is 'readable' (the secret Unix code meaning that it has an incoming connection) with the UnixEventRegistry. The UnixEventRegistry tells the Dispatcher that it needs to run when things are idle to check for OS events. When it notices that the file descriptor is readable, the UnixEventRegistry posts the event that the SockListenModule registered with it.
So, in order to have the SockListenModule wake up and process the event,
or even have the Dispatcher to call UnixEventRegistry when it's 'idle', we
have to call the Dispatcher somehow. The Dispatcher has a family of methods
for this, but we will use this one:
Dispatcher::dispatchEvent()
Here is how the dispatching loop will be added into the code:
} // in reverse order later. } while (cin); + while (!dispatcher.isQueueEmpty()) + { + dispatcher.dispatchEvent(); + } for (ListenerList::iterator i = listeners.begin(); i != listeners.end(); ++i) {
Now, we are calling the Dispatcher, which in turn calls UnixEventRegistry, which in turn posts other events to the Dispatcher on behalf of our various SockListenModules when an incoming connection can be accepted. But, we're still not doing anything with any of these incoming connections. Now, it's time to tackle that problem.
Sadly, this is probably the most complicated part of the program, and where you actually have to do polling.
The basic way you retrieve acceped connections is through the SockListenModule's `plug´. The plug has to be asked if there's a connection to read, and then the connection has to be read from it. Also, the only way to check for errors that may have occured on the listening socket is to ask the SockListenModule if any errors have occured. It has no way of signalling error conditions through events.
Both of these things require various parts of the SockListenModule to be polled periodically. This will have to happen in the dispatching loop. There will also be a method added to the Listener so that all the actual polling can happen in that method, keeping the dispatching loop fairly clean.
First though, you should go over here and read what a plug is.
There, now that you've done that, it should be apparent why we need to
'pull a plug' from SockListenModule and read connections from it. The added
method is named Listener::doPoll
, and it requires a new member
variable, lplug_
to be added to the Listener class.
lplug_
must initialied in the Listener::Listener
constructor. And finally, the code to scan through the listening modules and
poll each of them is added to the dispatching loop. Here is the final
code:
#include <StrMod/SockListenModule.h> #include <StrMod/SocketModule.h> #include <EHnet++/SocketAddress.h> #include <EHnet++/InetAddress.h> #include <UniEvent/Dispatcher.h> #include <UniEvent/SimpleDispatcher.h> #include <UniEvent/UnixEventRegistry.h> #include <UniEvent/UnixEventPoll.h> #include <UniEvent/RegistryDispatcherGlue.h> #include <iostream> #include <string> #include <list> using strmod::strmod::SockListenModule; using strmod::strmod::SocketModule; using strmod::unievent::Dispatcher; using strmod::unievent::UnixEventRegistry; // The EHnet++ library has not yet been converted to use namespaces. class Listener { public: // Lets alias a few types that we use all over in this class to make things // easier on our fingers. typedef ::strmod::ehnet::SocketAddress SocketAddress; typedef ::strmod::unievent::Dispatcher Dispatcher; typedef ::strmod::unievent::UnixEventRegistry UnixEventRegistry; Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg); ~Listener(); //! Called once a loop from the event dispatching loop. void doPoll(); private: SockListenModule *listener_; SockListenModule::SLPlug *lplug_; SocketAddress * const from_; SocketAddress * const to_; Dispatcher &disp_; UnixEventRegistry &ureg_; }; Listener::Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg) : listener_(NULL), lplug_(NULL), from_(from.Copy()), to_(to.Copy()), disp_(disp), ureg_(ureg) { listener_ = new SockListenModule(*from_, disp, ureg, 2); lplug_ = listener_->makePlug(); } Listener::~Listener() { if (lplug_) { listener_->deletePlug(lplug_); lplug_ = NULL; } delete listener_; delete from_; delete to_; } void Listener::doPoll() { if (listener_ && listener_->hasError()) { // If the listener has an error, print the error. { using std::cerr; ::memset(errbuf, '\0', sizeof(errbuf)); char errbuf[256]; // Magic number! listener_->getError().getErrorString(errbuf, sizeof(errbuf) - 1); cerr << "Listening socket has this error: (" << listener_->getError().getSyscallName() << ") " << errbuf << "\n"; } // Then tear down the listening socket and try to rebuild it. if (lplug_) { listener_->deletePlug(lplug_); lplug_ = NULL; } delete listener_; listener_ = NULL; } if (!listener_) { listener_ = new SockListenModule(*from_, disp_, ureg_, 2); lplug_ = listener_->makePlug(); } if (lplug_ && lplug_->isReadable()) { strmod::strmod::SocketModuleChunkPtr smcp = lplug_->getConnection(); SocketModule *incoming = smcp->getModule(); delete incoming; } } typedef std::list<Listener *> ListenerList; int main() { using ::strmod::lcore::U2Byte; using ::strmod::ehnet::InetAddress; strmod::unievent::SimpleDispatcher dispatcher; strmod::unievent::UnixEventPoll pollmanager(&dispatcher); strmod::unievent::RegistryDispatcherGlue glue(&dispatcher, &pollmanager); ListenerList listeners; using std::cin; using std::cout; using std::string; do { U2Byte inport = 0; string outaddr; unsigned int outport = 0; cin >> inport >> outaddr >> outport; if (cin) { InetAddress listenaddr(inport); InetAddress destaddr(outaddr, outport); Listener *listener = new Listener(listenaddr, destaddr, dispatcher, pollmanager); cout << "Accepting connections on port " << inport << " and redirecting data to a connection to " << outaddr << " port " << outport << "\n"; listeners.push_front(listener); // Reverse order so they get deleted } // in reverse order later. } while (cin); while (1) { dispatcher.dispatchEvent(); { // This is a technique for lowering the cost of the i != end statement const ListenerList::iterator end = listeners.end(); for (ListenerList::iterator i = listeners.begin(); i != end; ++i) { (*i)->doPoll(); } } } { // This is a technique for lowering the cost of the i != end statement const ListenerList::iterator end = listeners.end(); for (ListenerList::iterator i = listeners.begin(); i != end; ++i) { delete *i; *i = NULL; } } return 0; }
Well, that's over with now. On to picking it apart for what it doesn't do.
This program now properly accepts connections. And, as an added bonus, it also tries to resurrect dead listening sockets. Sadly, every time it gets a connection, it just closes it down. No outbound connections, no data forwarding. It's rather sad, really.
Now, to more explicitly enumerate what the program doesn't do yet, and how those things might be accomplished.
It does not make an outgoing connection in response to an incoming one. It doesn't forward data back and forth between the incoming connection and the non-existent (so far) outgoing connection. There is also one last thing it should do, and that's closing down the incoming and outgoing connections when they are no longer needed.
The first two are actually pretty trivial to solve. Here is some code that can be added into the Listeners::doPoll function that will fix the problem:
if (incoming) { SocketModule *outgoing = new SocketModule(*to_, disp_, ureg_, false); incoming->makePlug(0)->plugInto( *(outgoing->makePlug(0)) ); }
The magic of the UnixEventRegistry and the Dispatcher (which, as you noticed, are passed into the SocketModule constructor) keep the data moving between the two SocketModules after they've been connected through their plugs. It is, of course, not magic, but a detailed explanation is beyond the scope of ths tutorial.
But, this code has some obvious, serious memory leaks. Nobody manages the existence of incoming or outgoing. And, there is no management of error conditions, or what happens when a connection is close somewhere.
In order to make all this happen, we need another data structure that represents an active 'forwarding', essentially, an incoming/outgoing connection pair. Since memory management of this entity is also closely tied to error handling, which are handled using events, we won't have to do external memory management. In other words, the connection pair object can manage its own existence.
Here's some preliminary code for ForwardedConnection
:
class ForwardedConnection { public: ForwardedConnection(SocketModule &incoming, SocketModule &outgoing); ~ForwardedConnection(); private: SocketModule &incoming_; SocketModule &outgoing_; }; ForwardedConnection::ForwardedConnection(SocketModule &incoming, SocketModule &outgoing) : incoming_(incoming), outgoing_(outgoing) { incoming_.makePlug(0)->plugInto( *(outgoing_.makePlug(0)) ); } ForwardedConnection::~ForwardedConnection() { delete &incoming_; delete &outgoing_; }
This doesn't do much to actually fix the problem, but it does now provide a nice bucket for the values we would like to save. As stated previously, we will need to handle errors, and we will need to use events. SocketModule is derived from StreamFDModule and has an interface for requesting that events be posted in certain situations. Here are the the StreamFDModule methods:
void StreamFDModule::onErrorIn(ErrorType err, const unievent::EventPtr &ev);
And, here's the declaration for StreamFDModule::ErrorType
:
enum ErrorType { ErrRead, //!< Error while reading, might have read an EOF. Must be lowest enum value. ErrWrite, //!< Error while writing, might have written an EOF. ErrGeneral, //!< General error affecting both reading and writing. ErrFatal //!< General, fatal error affecting both reading and writing. Must be highest enum value. };
So, StreamFDModule lets us register an event to be posted when an error in a particular category occurs. And, that error includes an EOF condition. Since we want the forward to continue until the parties disconnect or there's some kind of error, this matches our needs perfectly.
Notice, that the interface wants an EventPtr. This means we need to create the event object ourselves, and that we need to dynamically allocate it. Since we're passing in a reference counted pointer, we essentially have joint ownership after we register it. That means we had best use our own reference counted pointer to keep track of it. Lastly, the event will need to call on of our own functions when triggered, so we'll need to create our own event class that knows what thing it needs to call back.
Here's a preliminary sketch of what the event class would look like:
class EerrorEvent : public strmod::unievent::Event { public: ErrorEvent(ForwardedConnection &parent) : parent_(parent), parentgood_(true) { } virtual ~ErrorEvent() {} virtual void triggerEvent(Dispatcher *dispatcher = 0) { if (parentgood_) { parent_.errorOccurred(); } } void parentGone() { parentgood_ = false; } private: ForwardedConnection &parent_; bool parentgood_; };
As you notice, the ErrorEvent object will need a pointer the the ForwardedConnection object it's associated with. Also, the ForwardedConnection will have to add an errorOccured method. Lastly, the ForwardConnection will have to call parentGone() when it disappears so the event won't do anything wrong if it is fired after the parent disappears.
Also, ErrorEvent is largely a vaguely named class that has an intimate relationship with ForwardedConnection. It's a utility class. I like to make those into nested classes because their names shouldn't be around polluting the namespace, and often, the class isn't even needed outside the class that uses it.
So, with these ideas in mind, here's a re-done ForwardedConnection and ErrorEvent:
class ForwardedConnection { public: ForwardedConnection(SocketModule &incoming, SocketModule &outgoing); ~ForwardedConnection(); private: class ErrorEvent : public strmod::unievent::Event { public: ErrorEvent(ForwardedConnection &parent) : parent_(parent), parentgood_(true) { } virtual ~ErrorEvent() {} virtual void triggerEvent(Dispatcher *dispatcher = 0) { if (parentgood_) { parent_.errorOccurred(); } } void parentGone() { parentgood_ = false; } private: ForwardedConnection &parent_; bool parentgood_; }; friend class ErrorEvent; SocketModule &incoming_; SocketModule &outgoing_; strmod::unievent::EventPtrT<ErrorEvent> errev_; void errorOccurred(); }; ForwardedConnection::ForwardedConnection(SocketModule &incoming, SocketModule &outgoing) : incoming_(incoming), outgoing_(outgoing), errev_(new ErrorEvent(*this)) { incoming_.makePlug(0)->plugInto( *(outgoing_.makePlug(0)) ); } ForwardedConnection::~ForwardedConnection() { errev_->parentGone(); delete &incoming_; delete &outgoing_; }
There, now the ErrorEvent will properly connect back to the parent, and
all the naming is OK and everything. Sadly, the parent doesn't yet have an
errorOccured()
method, nor does it arrange for the SocketModules
to post the error event when something happens.
The StreamFDModule (which SocketModule is derived from) method that lets
us register to have events posted is void onErrorIn(ErrorType err,
const unievent::EventPtr &ev)
. As you can see, it has several
error types or categories you can register for errors in. I want to register
for errors in all of those categories because they are all relevant. Here is
a rewrite of the ForwardedConnection constructor that accomplishes this:
ForwardedConnection::ForwardedConnection(SocketModule &incoming, SocketModule &outgoing) : incoming_(incoming), outgoing_(outgoing), errev_(new ErrorEvent(*this)) { using strmod::strmod::StreamFDModule; incoming_.makePlug(0)->plugInto( *(outgoing_.makePlug(0)) ); incoming_.onErrorIn(StreamFDModule::ErrRead, errev_); incoming_.onErrorIn(StreamFDModule::ErrWrite, errev_); incoming_.onErrorIn(StreamFDModule::ErrGeneral, errev_); incoming_.onErrorIn(StreamFDModule::ErrFatal, errev_); outgoing_.onErrorIn(StreamFDModule::ErrRead, errev_); outgoing_.onErrorIn(StreamFDModule::ErrWrite, errev_); outgoing_.onErrorIn(StreamFDModule::ErrGeneral, errev_); outgoing_.onErrorIn(StreamFDModule::ErrFatal, errev_); }
Yes, it is a lot of repetive code. You are not mistaken.
Now, all that's left is implementing
ForwardedConnect::errorOccurred()
. This can be a bit tricky if
we want to do it 'correctly'. Doing it correctly is not strictly necessary
for a functioning system. In this case, correctly means that connections are
not closed until they should be.
The main issue is 'half-open' connections. Since TCP connections are bi-directional, it is possible for a connection to be in a half-open state, so only one direction has data flowing. If you immediately close both connections on error, this state cannot exist on connections going through the forwarder. Some programs and protocols do require half-open connections to function, but most do not.
Here is an implementation of
ForwardedConnect::errorOccurred()
that does not hand half-open
connections properly:
void ForwardedConnect::errorOccurred() { delete this; }
As you see, the ForwardedConnection uses events to manage its own existence, and that's why it doesn't need to be put into a data structure that will manage its existence like the Listener objects need.
If you look at the ForwardedConnection destructor, you can see that it just deletes the SocketModules it's keeping track of, and tell its event that it's going away. Deleting the SocketModules has the effect of closing them.
If you put all this together now, you should have a largely functional port forwarder:
#include <StrMod/SockListenModule.h> #include <StrMod/SocketModule.h> #include <EHnet++/SocketAddress.h> #include <EHnet++/InetAddress.h> #include <UniEvent/Dispatcher.h> #include <UniEvent/SimpleDispatcher.h> #include <UniEvent/UnixEventRegistry.h> #include <UniEvent/UnixEventPoll.h> #include <UniEvent/RegistryDispatcherGlue.h> #include <iostream> #include <string> #include <list> using strmod::strmod::SockListenModule; using strmod::strmod::SocketModule; using strmod::unievent::Dispatcher; using strmod::unievent::UnixEventRegistry; // The EHnet++ library has not yet been converted to use namespaces, // so SocketAddress and InetAddress can be used 'bare'. class ForwardedConnection { public: ForwardedConnection(SocketModule &incoming, SocketModule &outgoing); ~ForwardedConnection(); private: class ErrorEvent : public strmod::unievent::Event { public: ErrorEvent(ForwardedConnection &parent) : parent_(parent), parentgood_(true) { } virtual ~ErrorEvent() {} virtual void triggerEvent(Dispatcher *dispatcher = 0) { if (parentgood_) { parent_.errorOccurred(); } } void parentGone() { parentgood_ = false; } private: ForwardedConnection &parent_; bool parentgood_; }; friend class ErrorEvent; SocketModule &incoming_; SocketModule &outgoing_; strmod::unievent::EventPtrT<ErrorEvent> errev_; void errorOccurred(); }; ForwardedConnection::ForwardedConnection(SocketModule &incoming, SocketModule &outgoing) : incoming_(incoming), outgoing_(outgoing), errev_(new ErrorEvent(*this)) { using strmod::strmod::StreamFDModule; incoming_.makePlug(0)->plugInto( *(outgoing_.makePlug(0)) ); incoming_.onErrorIn(StreamFDModule::ErrRead, errev_); incoming_.onErrorIn(StreamFDModule::ErrWrite, errev_); incoming_.onErrorIn(StreamFDModule::ErrGeneral, errev_); incoming_.onErrorIn(StreamFDModule::ErrFatal, errev_); outgoing_.onErrorIn(StreamFDModule::ErrRead, errev_); outgoing_.onErrorIn(StreamFDModule::ErrWrite, errev_); outgoing_.onErrorIn(StreamFDModule::ErrGeneral, errev_); outgoing_.onErrorIn(StreamFDModule::ErrFatal, errev_); } ForwardedConnection::~ForwardedConnection() { delete &incoming_; delete &outgoing_; errev_->parentGone(); } void ForwardedConnection::errorOccurred() { std::cerr << "Closing down a connection due to an error!\n"; delete this; } //---- class Listener { public: // Lets alias a few types that we use all over in this class to make things // easier on our fingers. typedef ::strmod::ehnet::SocketAddress SocketAddress; typedef ::strmod::unievent::Dispatcher Dispatcher; typedef ::strmod::unievent::UnixEventRegistry UnixEventRegistry; Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg); ~Listener(); //! Called once a loop from the event dispatching loop. void doPoll(); private: SockListenModule *listener_; SockListenModule::SLPlug *lplug_; SocketAddress * const from_; SocketAddress * const to_; Dispatcher &disp_; UnixEventRegistry &ureg_; }; Listener::Listener(const SocketAddress &from, const SocketAddress &to, Dispatcher &disp, UnixEventRegistry &ureg) : listener_(NULL), lplug_(NULL), from_(from.Copy()), to_(to.Copy()), disp_(disp), ureg_(ureg) { listener_ = new SockListenModule(*from_, disp, ureg, 2); lplug_ = listener_->makePlug(); } Listener::~Listener() { if (lplug_) { listener_->deletePlug(lplug_); lplug_ = NULL; } delete listener_; delete from_; delete to_; } void Listener::doPoll() { if (listener_ && listener_->hasError()) { // If the listener has an error, print the error. { using std::cerr; char errbuf[256]; // Magic number! ::memset(errbuf, '\0', sizeof(errbuf)); listener_->getError().getErrorString(errbuf, sizeof(errbuf) - 1); cerr << "Listening socket has this error: (" << listener_->getError().getSyscallName() << ") " << errbuf << "\n"; } // Then tear down the listening socket and try to rebuild it. if (lplug_) { listener_->deletePlug(lplug_); lplug_ = NULL; } delete listener_; listener_ = NULL; } if (!listener_) { listener_ = new SockListenModule(*from_, disp_, ureg_, 2); lplug_ = listener_->makePlug(); } if (lplug_ && lplug_->isReadable()) { strmod::strmod::SocketModuleChunkPtr smcp = lplug_->getConnection(); SocketModule *incoming = smcp->getModule(); if (incoming) { SocketModule *outgoing = new SocketModule(*to_, disp_, ureg_, false); new ForwardedConnection(*incoming, *outgoing); } } } typedef std::list<Listener *> ListenerList; int main() { using ::strmod::lcore::U2Byte; using ::strmod::ehnet::InetAddress; strmod::unievent::SimpleDispatcher dispatcher; strmod::unievent::UnixEventPoll pollmanager(&dispatcher); strmod::unievent::RegistryDispatcherGlue glue(&dispatcher, &pollmanager); ListenerList listeners; using std::cin; using std::cout; using std::string; do { U2Byte inport = 0; string outaddr; unsigned int outport = 0; cin >> inport >> outaddr >> outport; if (cin) { InetAddress listenaddr(inport); InetAddress destaddr(outaddr, outport); Listener *listener = new Listener(listenaddr, destaddr, dispatcher, pollmanager); cout << "Accepting connections on port " << inport << " and redirecting data to a connection to " << outaddr << " port " << outport << "\n"; listeners.push_front(listener); // Reverse order so they get deleted } // in reverse order later. } while (cin); while (1) { dispatcher.dispatchEvent(); { // This is a technique for lowering the cost of the i != end statement const ListenerList::iterator end = listeners.end(); for (ListenerList::iterator i = listeners.begin(); i != end; ++i) { (*i)->doPoll(); } } } { // This is a technique for lowering the cost of the i != end statement const ListenerList::iterator end = listeners.end(); for (ListenerList::iterator i = listeners.begin(); i != end; ++i) { delete *i; *i = NULL; } } return 0; }
Now, it might be nice to test this out on a Linux box. Here are some tests you can run. Commands here will be preceeded by '%' signs. Things you type will be in blue, and the output will be in black.
% g++3 -I ../libNet/include PortForward.cxx -o portforward -L ../libNet/generated/lib -lNet % (echo '7000 localhost 22' ; echo '7001 irc.openprojects.net 6667') 7000 localhost 22 7001 irc.openprojects.net 6667 % (echo '7000 localhost 22' ; echo '7001 irc.openprojects.net 6667') | ./portforward readevptr_ == 0x808f760 errorevptr_ == 0x8090118 Accepting connections on port 7000 and redirecting data to a connection to localhost port 22 readevptr_ == 0x8090f70 errorevptr_ == 0x8090f88 Accepting connections on port 7001 and redirecting data to a connection to irc.openprojects.net port 6667
In another window type:
% ssh -p 7000 localhost
You'll get an ssh session to your own computer (provided you are running an ssh daemon). If you start up an IRC client and point it to localhost, port 7001 as your server, it will act like it connects to irc.openprojects.net (provided of course, that you are connected to the Internet at large). You'll be able to start up multiple ssh sessions, and use them simultaneously with your IRC session.