PHP Text Replacement

fluiduk · 6 Nov 2006 at 17:15

Hi All

Working on a script that imports a large amount of data from a csv file into a database.

However some of the lines in the CSV contain nasty characters which mySQL doesn't like, apostraphes for example.

Can someone recommend a piece of code that can be used to replace these.

I have had a look at a lot of the examples online, however I need to be able to specify multiple rules, ie replace ' with a ft or & with and.

I dont want to use too many loops etc as the data file will have almost 80000 lines when the site goes live and I don't want it to take weeks. At the moment my test file has 7000 lines and it takes 3-4 seconds to process that.

Any advice is greatly appreciated.

Thanks
Aaron

PS - in case it helps, here is the script so far

Code:

$file = array();
foreach(file('****.csv') as $line)
{
    $file[] = explode(',', $line);
}


foreach($file as $data) {

mysql_select_db($database_wp_master, $wp_master);
$insert*** = "INSERT INTO ***_temp VALUES('$data[12]','$data[13]','$data[11]','$data[15]','$data[9]','$data[3]','$data[20]','$data[21]')";
mysql_query($insert***, $wp_master) or die(mysql_error());

}
echo 'Data Imported<br/><br/>';

fluiduk · 6 Nov 2006 at 17:47

Thanks Jestar

Im not too sure whats this sprintf stuff is and how it works, how would I rewrite my query to use it?

Would it be like this?

Code:

$query = sprintf("INSERT INTO **_temp VALUES('%s','%s','%s','%s','%s','%s','%s','%s'",
mysql_real_escape_string($data[12]),
mysql_real_escape_string($data[13]),
mysql_real_escape_string($data[11]),
mysql_real_escape_string($data[15]),
mysql_real_escape_string($data[9]),
mysql_real_escape_string($data[3]),
mysql_real_escape_string($data[20]),
mysql_real_escape_string($data[21]));
mysql_query($query, $wp_master) or die(mysql_error());

fluiduk · 6 Nov 2006 at 18:10

Got this working. Only issue I have...

Will this add slashes everywhere in my text?

Aaron

fluiduk · 6 Nov 2006 at 18:43

Would it be a good idea to use something else to remove all the slashes? Or should I remove the characters that are causing the slashes before I run the query, so that way I can cut out the slashes that occur frequently and still be safer from injection attacks etc.

The data all comes from a distributors CSV product list, so there is no user input as such - although it's best not too leave it to chance