Yet Another Concept of Custom Manageable FTP (Part II)

Hi! The article of the custom manageable FTP is now continue. To read previous part content, please click on this link (http://paparadit.blogspot.com/2012/04/yet-another-concept-of-custom.html). It will explained step-by-step about setting up virtual users on FTP server. To make your mind clear, if you have 10 FTP servers, the setting must installed in each machine.

A most tricky part is the authentication & uploader sub-routines. Before I explain it technically, here is a pseudo-concept:

  1. Provide users DB on a opt. DB Server.
  2. Provide a single HTTP Server for user authentication.
  3. Once a user has been successfully authenticated as well in front-end system, get the IP of FTP Server where they allowed to make a link.
  4. Also create immediately the FTP user using htpasswd feature and create all of the FTP environments (including home directory and it's permissions).
  5. Try to adding information to the file uploaded from client or some kind like that to securing it. Eg: a complete date string attached to file name will avoid it replaced by the same file or try to replace an empty space in file name into "_" characters, etc, etc.
  6. If there's file uploaded from user, make a direct stream connection to FTP server or IP address got from point #3, using ftp_connect, ftp_login and ftp_put (with additional FTP_AUTORESUME parameter).
  7. After file transferred successfully, move it to final destination.
  8. Freeing all buffer before quitting to save memory.

So, on this experiment, I currently using 3 MySQL tables; the user table, the ftp server mapping table & logging table. Take a look at below user table (m_users):


The table contain 3 fields only. The branch field indicate the unit code referring where the user come from. Create a dummy record on it like below example picture:


While ftp server mapping table only contains 2 fields (m_ftp).


The branch_grp field, contains 2 digit left of the unit code. It's described where the dedicated FTP will take place. Insert 3 dummy records as shown on below picture, as we'll use a real 10.6.1.19 machine:


The last table (m_log), purposed for transaction logging. Replacing a common /var/log/vsftpd.log file. The records will auto-inserted later by the system.


Next, let's create UI form. This form contains user ID, password and a file type object, also a submit button.

So, when submit button pressed, the system will executing authentication to database & creating FTP users in realtime mode. The creation of realtime FTP users will be made from SSH connections - automatically. So, normally we need libssh2.so and OpenSSL (http://www.php.net/manual/en/book.ssh2.php) natively installed on the web server. If your server luckily have it, test it with phpinfo files. It will displayed like below picture:


Anyway, if you found difficult to installed it manually, you'll no find anything on phpinfo file.


Since the installation of libssh2.so (and all of the dependencies) will takes time and too much complicated, so I'll using third party libraries which give the same basic function (and it's much easier). It named as phpseclib. Download it & place it to lib folder under the web server (/var/www/html/lib).

First, try the demo files. If it failed to upload a file to one of FTP server, then take a look at Linux firewall and SELinux configuration by running setup in console.


Even it show disabled, you still need to read /etc/selinux/config file exactly. In my experiments - weirdly - it's shows that the SELinux still in enforcing mode:


So, you must edit it to "disable":


Save it, reboot and make sure that the phpseclib demo file running successfully. Only if you're sure that all above tested and worked 100%, then, here below what belong on core PHP file (the front-end system):

<?
if (isset($_POST['SUBMIT']))
{
// DB initialization
define("DB_HOST", "db.host.or.ip");
define("DB_USER", "db_user");
define("DB_PASS", "db_pass");
define("DB_NAME", "db_ftp");
$link=mysql_connect(DB_HOST,DB_USER,DB_PASS);
mysql_select_db(DB_NAME);
$usr=$_POST['usr'];
$passwd=$_POST['passwd'];
if ($_FILES['filenya']['size']!=0) $imageexist=true;
else $imageexist=false;

// HTTP authentication
$r=mysql_query("select * from m_users where usr='$usr' and passwd='$passwd'");
$num=mysql_num_rows($r);
if (($num==1) && ($imageexist))
{
while ($row=mysql_fetch_array($r)) {
$branch=$row['branch']; }

$r=mysql_query("select * from m_ftp where branch_grp=left('$branch',2)");
while ($row=mysql_fetch_array($r)) {
$ip_ftp=$row['ip_ftp'];
$branch_grp='kan' . $row['branch_grp']; }

$ftp_user_name=$usr;
$ftp_user_pass=$passwd;
$ftp_server=$ip_ftp;
$imagefile=$_FILES['filenya']['tmp_name'];
$imagefile_name=strtolower($_FILES['filenya']['name']);
$imagefile_name=str_replace(' ','_',$imagefile_name);
$imagefile_name=date ('Y-m-d',mktime(date('H'),date('i'),date('s'),date('m'),date('d'),date('Y'))) . '_' . $imagefile_name;
$destination_file=$imagefile_name;

// create virtual users to selected FTP
include('lib/phpseclib/SSH2.php');
$ssh = new Net_SSH2($ftp_server);
if (!$ssh->login('root', 'ftp_svr_password')) exit('Login Failed');
$ssh->exec('htpasswd -b /etc/vsftpd/vsftpd.passwd ' . $ftp_user_name . ' ' . $ftp_user_pass);
$ssh->exec('mkdir /var/www/users/' . $ftp_user_name);
$ssh->exec('chown vsftpd:ftp_users /var/www/users/' . $ftp_user_name);

// connect, login, and transfer the file
$conn_id = ftp_connect($ftp_server);
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
$upload = ftp_put($conn_id, $destination_file, $imagefile, FTP_BINARY, FTP_AUTORESUME);

// check it if succeed
if (ftp_size($conn_id,$destination_file)!=0) {
mysql_query("insert into m_log values('$ftp_user_name','$destination_file',now())");
$ssh->exec('mv /var/www/users/' . $ftp_user_name . '/' . $destination_file . ' /var/www/users/' . $branch_grp);
echo("<script>alert('File $destination_file OK!')</script>"); }
else echo("<script>alert('Failed to upload $imagefile_name')</script>");

// freeing buffer
mysql_free_result($r);
$ssh->exec('htpasswd -D /etc/vsftpd/vsftpd.passwd ' . $ftp_user_name);
$ssh->exec('rmdir /var/www/users/' . $ftp_user_name);
$ssh->exec('exit');
ftp_close($conn_id);
}
else echo("<script>alert('Authentication failed or no file uploaded!')</script>");
}
?>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>FTP Concept</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
<form action="index.php" method="post" enctype="multipart/form-data" name="form1">
<table width="100%" border="0" cellspacing="1" cellpadding="1">
<tr bgcolor="#000000"> <td colspan="3">
<font color="#FFFFFF">&#8226; <strong><font size="2" face="Geneva, Arial, Helvetica, sans-serif">
Manageable FTP with PHP (&copy; 2012 by Eko Wahyudiharto)</font></strong></font></td> </tr>
<tr> <td width="11%"><font size="2" face="Geneva, Arial, Helvetica, sans-serif">User Name</font></td>
<td width="2%"><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td width="87%"><input name="usr" type="text" id="usr"></td> </tr>
<tr> <td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">Password</font></td>
<td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td><input name="passwd" type="password" id="passwd"></td> </tr>
<tr> <td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">File to Upload</font></td>
<td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td><input name="filenya" type="file" id="filenya"></td> </tr>
<tr> <td>&nbsp;</td> <td>&nbsp;</td>
<td><input type="submit" name="SUBMIT" value="Submit"></td> </tr> </table>
</form>
</body>
</html>

Save it and run it from browser. Try to upload a file and press submit button:


If there's nothing goes wrong, then my concept is worked perfectly. Take a look at picture below:


It explained that the realtime FTP user generator perfectly creating user P81000. But note that the files uploaded now has move to "kan03" folder (indicating that it could serve as it planned):


Last, take a look at m_log table, here's a record inserted when the file has successfully uploaded. Voila!:


Conclusion:
This concept is tested, worked perfectly and it's ready to use. Even you may add some others functionality to enhance the security it self or others, including the hit performance handle or another environments such as the hardware availability. Have a mayday and please tell me your story on a box below. Thanks.

Labels: , , ,

  Post a Comment

Yet Another Concept of Custom Manageable FTP (Part I)

Dealing with tens of FTP users on a single FTP server is a damn quite easy job since we don't need to maintain hardly those users. It's as simple as creating some users for login on a PC & takes a bit way to create another, delete it or just read the physical log file. But then, how about if we have thousands of dynamic-moving-around users with more than one dedicated FTP servers around us? Well, at least, we're going to need some robots that work 24/7 in order to take care whole of the system. But, unfortunately we're not live in Terminator era that supplies any of it, still.

From below picture, there's a different user accessing different dedicated FTP server. We're talking about huge of FTP users here that will make the log files tracking seems impossible. Hence, also to avoid users storing garbage files to another FTP servers.


Basically, the concept is easy-cake - theoretically, but it's relevant to work on it. First rule is; No one of that FTP servers are browseable from clients & I don't need to create also managed those thousands FTP users manually but I have list contains of it stored in a table of MySQL database (thatís why I use a HTTP authentication server ñ in PHP of course). So, if your condition fit to the situation I described, this article may solve your problem.

The key is to create virtual FTP users! Yes, generate users at runtime on a flat text file with htpasswd cli ñ a feature of HTTPD service. Even the system can delete that virtual account after session ends. On this scene, I use RHEL server with built-in FTP daemon called as VSFTPD - as claimed as a Very Secure FTP Daemon (yet another secure, fast & stable FTP server). Unfortunately, a great number of Linux distros has lack of one important VSFTPD module named as pam_pwdfile (described for Pluggable Authentication Module purposed for authenticating users within htpasswd), including to this RHEL OS.

So, I suggest you to download this library to the HTTP authentication server. I got it from RPMBone site (ftp://ftp.pbone.net/mirror/ftp.pld-linux.org/dists/1.0/updates/general/i586/pam_pwdfile-0.98-2.i586.rpm). Continue to install it:


After it successfully installed, now we need to configure /etc/vsftpd.conf just like below:

anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
nopriv_user=vsftpd
virtual_use_local_privs=YES
guest_enable=YES
user_sub_token=$USER
local_root=/var/www/users/$USER
chroot_local_user=YES
hide_ids=YES
guest_username=vsftpd
xferlog_enable=YES
xferlog_file=/var/log/vsftpd.log
xferlog_std_format=YES
log_ftp_protocol=YES
pam_service_name=vsftpd
userlist_enable=YES
listen=YES
check_shell=NO

Next, configuring PAM to check the passwd file for users. The file to configure is /etc/pam.d/vsftpd:

auth required pam_pwdfile.so pwdfile /etc/vsftpd/vsftpd.passwd
account required pam_permit.so

Simply remove everything else from the file except both above line. After editing, save it. Just to make sure your above configuration files is marked with green OK sign, restart the service twice.

#service vsftpd restart
Shutting down vsftpd:                                      [  OK  ]
Starting vsftpd for vsftpd:                                [  OK  ]
#service vsftpd restart
Shutting down vsftpd:                                      [  OK  ]
Starting vsftpd for vsftpd:                                [  OK  ]

Next, create the passwd file containing a dummy user. I named it as "testis" & I grouped this user to "ftp_users" group. So, we need to create that group before creating the users:

#groupadd ftp_users
#htpasswd -c /etc/vsftpd/vsftpd.passwd testis

To add later additional users to the file, the command is change to:

#htpasswd -b /etc/vsftpd/vsftpd.passwd virgin boobs

The user ID is "virgin" and the password is "boobs". Next, continue to create a physical local user that's used by the virtual users to authenticate:

#useradd -d /home/vsftpd -g ftp_users -m -s /bin/false vsftpd

Also, create user's home directory since VSFTPD doesn't do it automatically.

#mkdir /var/www/users/testis
#chown vsftpd:ftp_users testis

Finally, restart the VSFTPD service. Make it tested from client using FTP client.


That's it. Now we have a basic skeleton template of dynamically manageable FTP servers.


Still confuse? It looks 100% make sense if you just stay tune on this blog for upcoming part II article in the next couple weeks, I'll plan to share the rest of it. Thank's for reading & have a great day!

Labels: , , ,

  Post a Comment

A Stumble Between Cloud Drive

What exactly is a cloud drive? in my opinion - It’s a virtual disk drive placed on a wide network or internet. In other words, there’s no necessity to carry up a USB drive anymore, but everywhere you need is an internet connection. I can say that cloud drive is a small part of cloud computing or any kind of this. I thought it was departed from FTP link mechanism, while you need your own domain to get it start. But with cloud drive, everything looks more portable now.

There are 2 big players on cloud drive; Google (GDrive) and Microsoft (SkyDrive). While Google integrating it with Google Mail (GMail), Microsoft combined it with Microsoft Live. It’s a different, but similar. Okay anyone know that Gmail has 8GB storage to archives email on each account, it’s a more than enough to save 2 DVD drives on that way. Since it takes lot of time to use up the spaces, this probably boiled down as a new Google product inherited.



Officially – AFAIK - Google has integrated GDrive into Windows shell by a small kit named Gmail Drive (click the link to get it), while others OS supported by each hardware vendors (such Acer Aspire One for their netbook products which came with Linux) or with Firefox add-ons which is support multi-platform. Anyway - after installing it - your Windows Explorer now enhanced with supplement folder named with Gmail Drive. As it has the same behavior as disk drive, this Gmail Drive also has properties window. Just point to it, right click & select properties. See, it has the same size with your Gmail space.



By the way you can do above way after you have an internet link. Further, try to go into the drive, click the Gmail Drive & it’ll appear a login window. Enter your account, and then verification window will show, just like picture below.



To take example, try to copy & paste a file or folder. After it copied, the drive will send an email alert inform you that you or someone using your account had store a thing(s) on GDrive. Now, how about SkyDrive? It’s resemble with GDrive (with extra drive additional & extra space!), but the software kit was develop from 3rd party (not officially from Microsoft).



You can try to download the kit named by SDExplorer from this link. Install it (an extra drive will available named with SDExplorer) & try to log on with your Windows Live ID (create one if you doesn’t have).



Login to it & - as I say - you’ll have an extra 25GB drive space! Is any provider can supply more than this? However, this SkyDrive has the same operation treatment as GDrive, as it simply as copy, cut or paste.

Conclusion
From example on this article, no need to spare money to buy an extra USB disk, since you’ll have additional 33GB for free on the net accessed precisely from your Windows Explorer.



Those “virtual” drive above hit a bit benefit on me, it’s a secondary storage to store my micro files (like configuration files or something else) just in case if you forget to carry your USB disk. While it contains micro files, it doesn’t take long time to download it from the cloud drive. Share your comment below, thanks for reading & good day.

Labels: , , , , , ,

  Post a Comment

Setting Up Email in Nokia 5610

Changing personal handset is an exhausted activity, especially to the who which have hundreds of contacts, including tens of scheduling appointments & some of critical configurations such as email setting. Which is – in other words – you have to move it whether you like or not. As what I did in last days, finally I had migrate to Nokia brand name cellular phone replacing my old HP6365 windows based mobile phone. I though, it is the time for me to get back to common communication model, as I tired of running PDA with me for more than 3 years!

I was interested with 5610 since attracted to the dummy model shown at one of local cellular market when I heading for a new candidate phone unexpectedly. The criteria for the candidates was too easy: high Mpx camera, supported large external memory, small screen, ease of use, internet ready, multimedia dedicated & light weight. At the ends, this 5610 series defeated others nominees which was: 02 Stealth, HTC Touch & N73.

No big deal that there’s no smart categories phone (Symbian or Windows OS) since still I miss for old phone function; call, messaging (SMS, MMS, Email & Instant Messaging), camera & MP3. I’d love to pick this 5610 because it’s sexiest form factor. It has passed all of my qualifications either. It also fit in my hands & great pleasure with the slider. Although I was shocked moving in from PDA, especially the lost touch screen & different UI. Anyway, it took times to take over the control.

Moving contacts was my first problem since Address Book from Laptop Outlook Express couldn’t permitted it. But it solved with Bluetooth data transfer per 5 contacts manually from HP6365. It also works moving the appointments data. Even that I had to made correction with some records. The Instant Messaging feature is so easily to setup & there’s no great difficult over it. I could connecting to Yahoo Messenger as easy as I connect from PC. One big shot for me was configuring the Gmail (Google mail) services since it more bit different from PDA or PC. And here below is the action I’d done from 5610 to setting the POP & SMTP configuration:

1. Make sure that you have an active account from Gmail. Check the POP activation option from Setting link.
2. Also make sure that you have already GPRS services activation from your network provider.
3. Go to 5610 Messaging Menu & point it to Email Message under Message Settings.
4. Edit Mailbox preferences & create a name. Example: GMail
5. Entry My name with your real name.
6. Set Email address with your email address complete with domain. Example: user_name@gmail.com
7. in Login information setting, fill your existing user name (without domain) & password.
8. Next, go to Incoming mail setting.
9. Set Incoming POP3 server with pop.gmail.com.
10. Set Security with Secure port.
12. Set Port with 995 and back to previous menu.
13. next, go to Outgoing setting.
14. Set Outgoing SMTP server with smtp.gmail.com.
15. Set SMTP authentification to On.
16. Set your user name & password (without domain name).
17. Set Security with Secure port.
18. Set Port with 465 and back to previous menu.
19. Save configuration & you’re done.
19. Give a try to retrieve or send an email.

Note that the configuration above is only works for Gmail email services, hence there’s no guarantee for others POP3 services or others network provider works for the same setting (I use SimPATI pre-paid from Telkomsel Indonesia for the test). I provide the configuration above also without any screenshot since I don’t have already a Java snap application. Be an understanding for my short review of my 5610.

Labels: , , , ,

  Post a Comment