The Wahyudiharto's Blog

Labels: Gadget, Handphone, Review, Samsung, Tips Trik

Hi! The article of the custom manageable FTP is now continue. To read previous part content, please click on this link (http://paparadit.blogspot.com/2012/04/yet-another-concept-of-custom.html). It will explained step-by-step about setting up virtual users on FTP server. To make your mind clear, if you have 10 FTP servers, the setting must installed in each machine.

A most tricky part is the authentication & uploader sub-routines. Before I explain it technically, here is a pseudo-concept:

1. Provide users DB on a opt. DB Server.
2. Provide a single HTTP Server for user authentication.
3. Once a user has been successfully authenticated as well in front-end system, get the IP of FTP Server where they allowed to make a link.
4. Also create immediately the FTP user using htpasswd feature and create all of the FTP environments (including home directory and it's permissions).
5. Try to adding information to the file uploaded from client or some kind like that to securing it. Eg: a complete date string attached to file name will avoid it replaced by the same file or try to replace an empty space in file name into "_" characters, etc, etc.
6. If there's file uploaded from user, make a direct stream connection to FTP server or IP address got from point #3, using ftp_connect, ftp_login and ftp_put (with additional FTP_AUTORESUME parameter).
7. After file transferred successfully, move it to final destination.
8. Freeing all buffer before quitting to save memory.

So, on this experiment, I currently using 3 MySQL tables; the user table, the ftp server mapping table & logging table. Take a look at below user table (m_users):


The table contain 3 fields only. The branch field indicate the unit code referring where the user come from. Create a dummy record on it like below example picture:


While ftp server mapping table only contains 2 fields (m_ftp).


The branch_grp field, contains 2 digit left of the unit code. It's described where the dedicated FTP will take place. Insert 3 dummy records as shown on below picture, as we'll use a real 10.6.1.19 machine:


The last table (m_log), purposed for transaction logging. Replacing a common /var/log/vsftpd.log file. The records will auto-inserted later by the system.


Next, let's create UI form. This form contains user ID, password and a file type object, also a submit button.

So, when submit button pressed, the system will executing authentication to database & creating FTP users in realtime mode. The creation of realtime FTP users will be made from SSH connections - automatically. So, normally we need libssh2.so and OpenSSL (http://www.php.net/manual/en/book.ssh2.php) natively installed on the web server. If your server luckily have it, test it with phpinfo files. It will displayed like below picture:


Anyway, if you found difficult to installed it manually, you'll no find anything on phpinfo file.


Since the installation of libssh2.so (and all of the dependencies) will takes time and too much complicated, so I'll using third party libraries which give the same basic function (and it's much easier). It named as phpseclib. Download it & place it to lib folder under the web server (/var/www/html/lib).

First, try the demo files. If it failed to upload a file to one of FTP server, then take a look at Linux firewall and SELinux configuration by running setup in console.


Even it show disabled, you still need to read /etc/selinux/config file exactly. In my experiments - weirdly - it's shows that the SELinux still in enforcing mode:


So, you must edit it to "disable":


Save it, reboot and make sure that the phpseclib demo file running successfully. Only if you're sure that all above tested and worked 100%, then, here below what belong on core PHP file (the front-end system):

<?
if (isset($_POST['SUBMIT']))
{
// DB initialization
define("DB_HOST", "db.host.or.ip");
define("DB_USER", "db_user");
define("DB_PASS", "db_pass");
define("DB_NAME", "db_ftp");
$link=mysql_connect(DB_HOST,DB_USER,DB_PASS);
mysql_select_db(DB_NAME);
$usr=$_POST['usr'];
$passwd=$_POST['passwd'];
if ($_FILES['filenya']['size']!=0) $imageexist=true;
else $imageexist=false;

// HTTP authentication
$r=mysql_query("select * from m_users where usr='$usr' and passwd='$passwd'");
$num=mysql_num_rows($r);
if (($num==1) && ($imageexist))
{
while ($row=mysql_fetch_array($r)) {
$branch=$row['branch']; }

$r=mysql_query("select * from m_ftp where branch_grp=left('$branch',2)");
while ($row=mysql_fetch_array($r)) {
$ip_ftp=$row['ip_ftp'];
$branch_grp='kan' . $row['branch_grp']; }

$ftp_user_name=$usr;
$ftp_user_pass=$passwd;
$ftp_server=$ip_ftp;
$imagefile=$_FILES['filenya']['tmp_name'];
$imagefile_name=strtolower($_FILES['filenya']['name']);
$imagefile_name=str_replace(' ','_',$imagefile_name);
$imagefile_name=date ('Y-m-d',mktime(date('H'),date('i'),date('s'),date('m'),date('d'),date('Y'))) . '_' . $imagefile_name;
$destination_file=$imagefile_name;

// create virtual users to selected FTP
include('lib/phpseclib/SSH2.php');
$ssh = new Net_SSH2($ftp_server);
if (!$ssh->login('root', 'ftp_svr_password')) exit('Login Failed');
$ssh->exec('htpasswd -b /etc/vsftpd/vsftpd.passwd ' . $ftp_user_name . ' ' . $ftp_user_pass);
$ssh->exec('mkdir /var/www/users/' . $ftp_user_name);
$ssh->exec('chown vsftpd:ftp_users /var/www/users/' . $ftp_user_name);

// connect, login, and transfer the file
$conn_id = ftp_connect($ftp_server);
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
$upload = ftp_put($conn_id, $destination_file, $imagefile, FTP_BINARY, FTP_AUTORESUME);

// check it if succeed
if (ftp_size($conn_id,$destination_file)!=0) {
mysql_query("insert into m_log values('$ftp_user_name','$destination_file',now())");
$ssh->exec('mv /var/www/users/' . $ftp_user_name . '/' . $destination_file . ' /var/www/users/' . $branch_grp);
echo("<script>alert('File $destination_file OK!')</script>"); }
else echo("<script>alert('Failed to upload $imagefile_name')</script>");

// freeing buffer
mysql_free_result($r);
$ssh->exec('htpasswd -D /etc/vsftpd/vsftpd.passwd ' . $ftp_user_name);
$ssh->exec('rmdir /var/www/users/' . $ftp_user_name);
$ssh->exec('exit');
ftp_close($conn_id);
}
else echo("<script>alert('Authentication failed or no file uploaded!')</script>");
}
?>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>FTP Concept</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body>
<form action="index.php" method="post" enctype="multipart/form-data" name="form1">
<table width="100%" border="0" cellspacing="1" cellpadding="1">
<tr bgcolor="#000000"> <td colspan="3">
<font color="#FFFFFF">&#8226; <strong><font size="2" face="Geneva, Arial, Helvetica, sans-serif">
Manageable FTP with PHP (&copy; 2012 by Eko Wahyudiharto)</font></strong></font></td> </tr>
<tr> <td width="11%"><font size="2" face="Geneva, Arial, Helvetica, sans-serif">User Name</font></td>
<td width="2%"><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td width="87%"><input name="usr" type="text" id="usr"></td> </tr>
<tr> <td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">Password</font></td>
<td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td><input name="passwd" type="password" id="passwd"></td> </tr>
<tr> <td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">File to Upload</font></td>
<td><font size="2" face="Geneva, Arial, Helvetica, sans-serif">:</font></td>
<td><input name="filenya" type="file" id="filenya"></td> </tr>
<tr> <td>&nbsp;</td> <td>&nbsp;</td>
<td><input type="submit" name="SUBMIT" value="Submit"></td> </tr> </table>
</form>
</body>
</html>

Save it and run it from browser. Try to upload a file and press submit button:


If there's nothing goes wrong, then my concept is worked perfectly. Take a look at picture below:


It explained that the realtime FTP user generator perfectly creating user P81000. But note that the files uploaded now has move to "kan03" folder (indicating that it could serve as it planned):


Last, take a look at m_log table, here's a record inserted when the file has successfully uploaded. Voila!:


Conclusion:
This concept is tested, worked perfectly and it's ready to use. Even you may add some others functionality to enhance the security it self or others, including the hit performance handle or another environments such as the hardware availability. Have a mayday and please tell me your story on a box below. Thanks.

Labels: FTP, Linux, Setting, Tips Trik

Dealing with tens of FTP users on a single FTP server is a damn quite easy job since we don't need to maintain hardly those users. It's as simple as creating some users for login on a PC & takes a bit way to create another, delete it or just read the physical log file. But then, how about if we have thousands of dynamic-moving-around users with more than one dedicated FTP servers around us? Well, at least, we're going to need some robots that work 24/7 in order to take care whole of the system. But, unfortunately we're not live in Terminator era that supplies any of it, still.

From below picture, there's a different user accessing different dedicated FTP server. We're talking about huge of FTP users here that will make the log files tracking seems impossible. Hence, also to avoid users storing garbage files to another FTP servers.


Basically, the concept is easy-cake - theoretically, but it's relevant to work on it. First rule is; No one of that FTP servers are browseable from clients & I don't need to create also managed those thousands FTP users manually but I have list contains of it stored in a table of MySQL database (thatís why I use a HTTP authentication server ñ in PHP of course). So, if your condition fit to the situation I described, this article may solve your problem.

The key is to create virtual FTP users! Yes, generate users at runtime on a flat text file with htpasswd cli ñ a feature of HTTPD service. Even the system can delete that virtual account after session ends. On this scene, I use RHEL server with built-in FTP daemon called as VSFTPD - as claimed as a Very Secure FTP Daemon (yet another secure, fast & stable FTP server). Unfortunately, a great number of Linux distros has lack of one important VSFTPD module named as pam_pwdfile (described for Pluggable Authentication Module purposed for authenticating users within htpasswd), including to this RHEL OS.

So, I suggest you to download this library to the HTTP authentication server. I got it from RPMBone site (ftp://ftp.pbone.net/mirror/ftp.pld-linux.org/dists/1.0/updates/general/i586/pam_pwdfile-0.98-2.i586.rpm). Continue to install it:


After it successfully installed, now we need to configure /etc/vsftpd.conf just like below:

anonymous_enable=NO
local_enable=YES
write_enable=YES
local_umask=022
nopriv_user=vsftpd
virtual_use_local_privs=YES
guest_enable=YES
user_sub_token=$USER
local_root=/var/www/users/$USER
chroot_local_user=YES
hide_ids=YES
guest_username=vsftpd
xferlog_enable=YES
xferlog_file=/var/log/vsftpd.log
xferlog_std_format=YES
log_ftp_protocol=YES
pam_service_name=vsftpd
userlist_enable=YES
listen=YES
check_shell=NO

Next, configuring PAM to check the passwd file for users. The file to configure is /etc/pam.d/vsftpd:

auth required pam_pwdfile.so pwdfile /etc/vsftpd/vsftpd.passwd
account required pam_permit.so

Simply remove everything else from the file except both above line. After editing, save it. Just to make sure your above configuration files is marked with green OK sign, restart the service twice.

#service vsftpd restart
Shutting down vsftpd:                                      [  OK  ]
Starting vsftpd for vsftpd:                                [  OK  ]
#service vsftpd restart
Shutting down vsftpd:                                      [  OK  ]
Starting vsftpd for vsftpd:                                [  OK  ]

Next, create the passwd file containing a dummy user. I named it as "testis" & I grouped this user to "ftp_users" group. So, we need to create that group before creating the users:

#groupadd ftp_users
#htpasswd -c /etc/vsftpd/vsftpd.passwd testis

To add later additional users to the file, the command is change to:

#htpasswd -b /etc/vsftpd/vsftpd.passwd virgin boobs

The user ID is "virgin" and the password is "boobs". Next, continue to create a physical local user that's used by the virtual users to authenticate:

#useradd -d /home/vsftpd -g ftp_users -m -s /bin/false vsftpd

Also, create user's home directory since VSFTPD doesn't do it automatically.

#mkdir /var/www/users/testis
#chown vsftpd:ftp_users testis

Finally, restart the VSFTPD service. Make it tested from client using FTP client.


That's it. Now we have a basic skeleton template of dynamically manageable FTP servers.


Still confuse? It looks 100% make sense if you just stay tune on this blog for upcoming part II article in the next couple weeks, I'll plan to share the rest of it. Thank's for reading & have a great day!

Labels: FTP, Linux, Setting, Tips Trik

Don't have idea why I'm so addicted with foursquare. It's so fun! Checked-in here, checked-in there and gaining some badges and points, that's all. Actually, nothing positive advantages from it except marked specific location and collecting badges. Even we need to spend charge to make internet connection. But somehow, this badges makes me curious. Here below my foursquare badges collection (I hope, the foursquare owner will provide goods from points replacement or badges - an iPad is a good start :)




That's my 20 badges after all. And I still looking for another below visible badges:




There are exactly total of 35 active badges available on foursquare. And here below the rest of 5 badges that's not visible for me to check-in :( (since it's so far far far far away)




So, how about you, guys?

Labels: Foursquare, My Tricks, Tips Trik, Weekend