WXforum.net

Web Weather => Weather Website PHP/AJAX scripting => Topic started by: zmarfak on July 08, 2020, 03:14:13 AM

Title: USGS translate array for quake-json.php
Post by: zmarfak on July 08, 2020, 03:14:13 AM
Hi,
depending on your location and settings in your scripts, this array is different for everyone.
I like to share mine, if somebody else need the same names you can copy - paste it, add it to your own.
From my Settings.php file, current entries :
Code: [Select]
$SITE['USGStranslate'] = array(
 'P?hala' => 'Pāhala',
 'Or?ova' => 'Orşova',
 'Ger?sh' => 'Gerāsh',
 'Qarn?b?d' => 'Qarnābād',
 'Bojn?rd' => 'Bojnūrd',   
);
Title: Re: USGS translate array for quake-json.php
Post by: zmarfak on July 24, 2020, 02:05:53 AM
Guys,
I have the impression that those headers at USGS get worse. (?)
Today I had following (before correction) : see picture 1
Maybe it was only my page? Therefore I checked it on the webpage of USGS : same , see picture 2

Are these headings created automatically ? I cannot imagine somebody types in those words.

This is getting stupid , my list of exceptions is growing in my Settings.php file, even duplicates for the same name. (and I hope I get the correct names)
Has anyone an explanation for this, or is it just me ?
Code: [Select]
$SITE['USGStranslate'] = array(
 'P?hala' => 'Pāhala',
 'Or?ova' => 'Orşova',
 'Ger?sh' => 'Gerāsh',
 'Qarn?b?d' => 'Qarnābād',
 'Bojn?rd' => 'Bojnūrd',
 'Pólis' => 'Polis',   
 'Argostólion' => 'Argostoli',
 'Karpenísi' => 'Karpenisi',
 'Álftanes' => 'Alftanes',
 'Argostólion' => 'Argostoli',
 'Karpenísi' => 'Karpenisi',
 'Álftanes' => 'Alftanes', 
 'Kontokáli' => 'Kontokáli',   
);

 
Title: Re: USGS translate array for quake-json.php
Post by: saratogaWX on July 24, 2020, 02:35:48 PM
Looking at the raw data https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us7000ard2.geojson for that one event in Iceland shows
Quote
Content-Security-Policy-Report-Only: worker-src 'none'; report-uri about:blank
X-Firefox-Spdy: h2
access-control-allow-headers: accept,origin,authorization,content-type
access-control-allow-methods: *
access-control-allow-origin: *
age: 19
cache-control: public, max-age=60
content-encoding: gzip
content-length: 1731
content-type: application/json; charset=utf-8
date: Fri, 24 Jul 2020 18:20:36 GMT
expires: Fri, 24 Jul 2020 18:21:36 GMT
last-modified: Fri, 24 Jul 2020 18:20:36 GMT
server: Apache
strict-transport-security: max-age=31536000
vary: Accept-Encoding
via: 1.1 2de9b6504a97ad8423645370927ef0cf.cloudfront.net (CloudFront)
x-amz-cf-id: xIkmSLFCKoqt-Bjm2hMLZ1QcMw6292518xMofYCKuqQBaFaQiD16Gw==
x-amz-cf-pop: SFO20-C1
x-cache: Hit from cloudfront
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1
which indicates that UTF-8 should be the response, but the content is 'mixed' for the legend
Quote
{
  "type": "Feature",
  "properties": {
    "mag": 4.5,
    "place": "8 km WSW of Álftanes, Iceland",
    "time": 1595201774551,
    "updated": 1595254187156,
    "tz": null,
    "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us7000ard2",
    "felt": 7,
    "cdi": 3.8,
    "mmi": null,
    "alert": null,
    "status": "reviewed",
    "tsunami": 0,
    "sig": 314,
    "net": "us",
    "code": "7000ard2",
    "ids": ",us7000ard2,",
    "sources": ",us,",
    "types": ",dyfi,origin,phase-data,",
    "nst": null,
    "dmin": 0.776,
    "rms": 0.65,
    "gap": 100,
    "magType": "mb",
    "type": "earthquake",
    "title": "M 4.5 - 8 km WSW of Álftanes, Iceland",
...

They've got some very funky character set translations going on in their geographical placename database.  :shock:  #-o
Title: Re: USGS translate array for quake-json.php
Post by: saratogaWX on July 24, 2020, 11:58:16 PM
I sent a note via the USGS Contact form
Quote
Hi,

Looking at https://earthquake.usgs.gov/earthquakes/eventpage/us7000ard2/executive
and the associated GeoJSON for the event at
https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us7000ard2.geojson

shows the place/title with characters like:

    "place": "8 km WSW of Álftanes, Iceland",

    "title": "M 4.5 - 8 km WSW of Álftanes, Iceland",

and this happens on other international place names as well.  While the page and GeoJSON both are returned
with UTF-8 content type indicated, I think there is some character set conversion issue in the USGS place name
database that is causing this odd problem.

Thanks in advance for your attention to the issue.

--
Best regards,
Ken
and received the reply:
Quote
U.S. Geological Survey Response [Ticket#148386] Odd characters in  GeoJSON and webpage text for earthquake

Thank you for contacting the USGS.

There is an issue with non-English characters. It is a known problem, that the Earthquake Hazards Team is working on a solution. Its a problem with the diacritical marks.

Please feel free to contact me if you have any additional questions

Mitch Adelson
USGS Menlo Park, CA

so, at least, they are aware of the issue and likely working on it.  If they fix the issue and use UTF-8 characters fully, then the entries in the $USGStranslate array will no longer be used or needed.
Title: Re: USGS translate array for quake-json.php
Post by: zmarfak on July 25, 2020, 01:55:20 AM
Ken,
thanks, I also have send a message just before I read this. :grin:
Thanks for doing this, at least we know now.
Title: Re: USGS translate array for quake-json.php
Post by: saratogaWX on July 28, 2020, 02:25:22 PM
Looks like the issue has gotten worse at the USGS site, both the GeoJSON and their webpage are overflowing with odd characters for the location name.

See https://earthquake.usgs.gov/earthquakes/eventpage/hv72054057/executive for an extreme example.

I've replied to the incident notification with that new information.  Hope they get it fixed soon.
Title: Re: USGS translate array for quake-json.php
Post by: zmarfak on July 29, 2020, 03:03:28 AM
Looks like the issue has gotten worse at the USGS site, both the GeoJSON and their webpage are overflowing with odd characters for the location name.

Yes, I noticed it too. This is getting annoying.
Title: Re: USGS translate array for quake-json.php
Post by: Silversword on July 31, 2020, 01:41:54 PM
Whooo!

Looks like USGS folks got it fixed.  Looks like our Hawaiian names are now good to go.

Thanks Ken for reporting this to them.

Aloha,

--Stan Y.
   Maui, Hawai'i
Title: Re: USGS translate array for quake-json.php
Post by: saratogaWX on July 31, 2020, 02:03:15 PM
One final fix on your site.  To wxquake.php change
Code: [Select]
include("top.php");
to
Code: [Select]
$useUTF8 = true; // force UTF-8 display
include("top.php");
to have it correctly display the 'new' UTF-8 characters in the page display.
Title: Re: USGS translate array for quake-json.php
Post by: Silversword on July 31, 2020, 02:17:46 PM
Hi Ken,

Will do that and thanks again for all the fixes that come your way.

Be safe over there.

Mahalo and Aloha for all that you do for us.

--Stan Y.
   Maui, Hawai'i
Title: Re: USGS translate array for quake-json.php
Post by: Silversword on July 31, 2020, 02:23:34 PM
Got it done and seems to be working.

Thanks again for all.

Aloha,

--Stan Y.
   Maui, Hawai'i
Title: Re: USGS translate array for quake-json.php
Post by: zmarfak on August 01, 2020, 01:56:37 AM
Looks like USGS folks got it fixed.  Looks like our Hawaiian names are now good to go.

Yes, here also everything ok