So extrahieren Sie Schlüsselwörter aus Bengali Text mit PHP

Question

Ich möchte Schlüsselwörter automatisch aus Bengali Textdateien mit PHP extrahieren. Ich habe diesen Code zum Lesen einer Bengali-Textdatei.

<?php 
$target_path = $_FILES['uploadedfile']['name']; 
header('Content-Type: text/plain;charset=utf-8'); 
$fp = fopen($target_path, 'r') or die("Can't open CEDICT."); 
$i = 0; 
while ($line = fgets($fp, 1024)) 
    { 
     print $line; 
     $i++; 
    } 
fclose($fp) or die("Can't close file."); 

Und ich fand folgende Codes, um die meisten 10 Schlüsselwörter zu extrahieren, aber es funktioniert nicht für bengalische Texte. Welche Änderungen sollte ich vornehmen?

function extractCommonWords($string){ 
     $stopWords = array('i','a','about','an','and','are','as','at','be','by','com','de','en','for','from','how','in','is','it','la','of','on','or','that','the','this','to','was','what','when','where','who','will','with','und','the','www'); 

     $string = preg_replace('/\s\s+/i', '', $string); // replace whitespace 
     $string = trim($string); // trim the string 
     $string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string); // only take alphanumerical characters, but keep the spaces and dashes too… 
     $string = strtolower($string); // make it lowercase 

     preg_match_all('/\b.*?\b/i', $string, $matchWords); 
     $matchWords = $matchWords[0]; 

     foreach ($matchWords as $key=>$item) { 
      if ($item == '' || in_array(strtolower($item), $stopWords) || strlen($item) <= 3) { 
       unset($matchWords[$key]); 
      } 
     } 
     $wordCountArr = array(); 
     if (is_array($matchWords)) { 
      foreach ($matchWords as $key => $val) { 
       $val = strtolower($val); 
       if (isset($wordCountArr[$val])) { 
        $wordCountArr[$val]++; 
       } else { 
        $wordCountArr[$val] = 1; 
       } 
      } 
     } 
     arsort($wordCountArr); 
     $wordCountArr = array_slice($wordCountArr, 0, 10); 
     return $wordCountArr; 
} 

Bitte helfen :(

Answer 1

sollten Sie einfach Änderungen vornehmen:

• ersetzen Stoppwörter in $stopWords Array mit der richtigen Bengali Stoppwörter
• diese Zeichenfolge entfernen $string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string); weil Bengali sybmols nicht überein Dieses Muster

Voll Code wie folgt aussieht:

<?php 

function extractCommonWords($string){ 
    // replace array below with proper Bengali stopwords 
    $stopWords = array('i','a','about','an','and','are','as','at','be','by','com','de','en','for','from','how','in','is','it','la','of','on','or','that','the','this','to','was','what','when','where','who','will','with','und','the','www'); 

    $string = preg_replace('/\s\s+/i', '', $string); // replace whitespace 
    $string = trim($string); // trim the string 
    // remove this preg_replace because Bengali sybmols doesn't match this pattern 
    // $string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string); // only take alphanumerical characters, but keep the spaces and dashes too… 
    $string = strtolower($string); // make it lowercase 

    preg_match_all('/\s.*?\s/i', $string, $matchWords); 
    $matchWords = $matchWords[0]; 

    foreach ($matchWords as $key=>$item) { 
     if ($item == '' || in_array(strtolower(trim($item)), $stopWords) || strlen($item) <= 3) { 
      unset($matchWords[$key]); 
     } 
    } 
    $wordCountArr = array(); 
    if (is_array($matchWords)) { 
     foreach ($matchWords as $key => $val) { 
      $val = trim(strtolower($val)); 
      if (isset($wordCountArr[$val])) { 
       $wordCountArr[$val]++; 
      } else { 
       $wordCountArr[$val] = 1; 
      } 
     } 
    } 
    arsort($wordCountArr); 
    $wordCountArr = array_slice($wordCountArr, 0, 10); 
    return $wordCountArr; 
} 

$string = <<<EOF 
টিপ বোঝে না, টোপ বোঝে না টিপ বোঝে না, কেমন বাপু লোক 
EOF; 
var_dump(extractCommonWords($string), $string); 

ausgegeben:

array(4) { 
    ["বোঝে"]=> 
    int(2) 
    ["টোপ"]=> 
    int(1) 
    ["না"]=> 
    int(1) 
    ["কেমন"]=> 
    int(1) 
} 
string(127) "টিপ বোঝে না, টোপ বোঝে না টিপ বোঝে না, কেমন বাপু লোক"