Solving some issues with Japanese and Chinese characters in files

Note to self and reader: This article may not be complete nor completely accurate, I'll try to complete and correct it if it's not, if I'm bothered to do so...

Note to self 2: Use < strong > in Html, not [b]. This is not a forum. And it's the triangle one, not the square bracket.

Lately... there's been nothing to write...

My page (Note: Page, not blog, i don't acknowledge it as one) is more about interesting tech things that i discover, or unique findings on Anime stuff, and not about my daily life. About things that you cannot find/hard to find someplace else, things that can be useful to some readers. Not a news blog either; somebody (the guy with the news obviously) would've written a better article about it. Reviews? Maybe, but that would be for my real pages that will never be completed and is currently floating somewhere in the fifth dimension in my head. And who would care about my life anyway? (The tinnitus ones are an exception, I just needed to rant (and tell the world something obvious))

So, it's hard to find something to write. Tricks like the ones I've previously posted don't just come to me, they piled up over the years and in one shot I've already posted a large majority of them.

That's why, don't treat this page like it'd be updated like every 1~2 days, even every week, can? Treat it more like the front page of silentpcreview.com, which only gets updated when something really interesting happens. I'd prefer that to Tom's Hardware, most of which articles I don't even read in-depth.

But, if you think that this page is useful, thank you for thinking that way.

Anyway, on with the main topic of the day...

Solving some issues with Japanese and Chinese characters in files

So you've downloaded a Japanese/Chinese song off the internet. However, when you play the file, what you see (in your player) is ‘šŒŽ‚Í‚é‚©. And when you want to burn that file with Nero, you see 472458~1. Or the best, you know how to make it appear right in Nero or other software. But when you burn it/extract it you still get Ë®˜ò¤«¤ª¤ê. Or the program simply rejects the file.

Or your non-English program just crashes, or you see no text.

This post (article?) will feature some of the issues you'd be likely to face if you ever come across this two non-English East-Asian languages. (Is there a geek that isn't an otaku?) In the case that you didn't know what to do then this post would be useful to you.

Situation 1: Burning Japanese/Chinese File Name In Nero

This one has been asked and replied a few times, and with my policy I'll cut it short.

Summary: Regional and Language Options

But there's something else that's much more convenient to use:

Useful Tool 1: Microsoft AppLocale

This is THE program most often used by many Chinese who are playing fan-translated Japanese games (eroge or not). It does the same thing as the Regional and Language Options in the Control Panel, but only changes the just for that program and for that session, and you don't have to restart the computer for that, meaning you can play a Japanese game now, then a Chinese game next, then do a disc quality scan without CDSpeed becoming Chinese. And much more convenient if you're testing which language the program will run with.

And the best part is it can also create shortcuts. Neat, huh?

Here is a good guide showing how works. The program in this case is Nero, hence this is also the solution to burning Chinese (or Japanese) filenames in Nero. But remember, this can also be used in many other programs to solve the same or similar problems.

http://forums.vr-zone.com/showthread.php?t=84923

There are cases where this won't work though. The most probable case is that the program is just a frontend/launcher for another non-English, non-Unicode program, hence even if you change the language of the launcher you won't be able to change that of the linked program. So unless you want to hex-edit you file such that it points to an AppLocale Shortcut, you'd have to use Regional and Language Options. Regional and Language Options is also slightly "more stable" in the sense of greater consistency, compatability, and less crashing. But all your Unicode programs will have their language changed.

Since the title is burning chinese filename in Nero, might as well go on about that.

Sometimes, when you burn in Nero, you will either get 1) Nero complaining that the filename/directory path is too long, or 2) It shows up fine in Nero but after burning, the filenames are Ë®˜ò¤«¤ª¤ê-ed. In which case,

Nero: Burning in UDF format

UDF stands for Universal Disk (or disc? Can't remember) Format. It has way less limitations compared to the ISO standard(s) - "s" denotes the various updates that came later. So, instead of tweaking with the ISO settings like some other guides may tell you to, why not try the UDF format? It also solves the problem with long filenames/directory paths, thus killing two bird with one stone.

The downside is, you won't be able to use the disc with Win98 and earlier systems. But these system will not properly support Unicode, Japanese and Chinese, and long file names anyway. So it's not exactly a loss.

Situation 2: Playing a Japanese/Chinese-subbed Japanese Game And The Text Doesn't Come Up

Note that this isn't "Playing a Japanese/Chinese-subbed Japanese Game And The Text is Ë®˜ò¤«¤ª¤ê". You would've figured that it can be solved by using AppLocale. Or at least tried AppLocale and succeeded.

In this scenario, even if you try various languages in AppLocale, the text just doesn't come up.

While this may be possible because your system doesn't have the necessary Chinese fonts installed (usually not the case, and usually the system will substitute the words that are not in the font with another font), this usually occurs because you don't have the necessary Japanese font installed. Yes, this is also the cause of Chinese words not showing up, ironically.

Useful Tool 2: jpfont.ttf

For this, find and download a file called jpfont.ttf, it should be ~2.8MB.

(In case you really suck at search engines:
http://search.yahoo.com/search?p=jpfont.ttf&fr=yfp-t-331&toggle=1&cop=mss&ei=UTF-8&fp_ip=SG&vc=, click on the first one)

Hopefully that solves the problem. Even if it didn't, well, it was worth a shot. And at least you solved another problem in advance.

Situation 3: Playing The MP3 File In Windows Media Player Shows ‘šŒŽ‚Í‚é‚©
(Or other media types in other players)

Yes, this can be solved using the AppLocale or Regional and Language Options, but all you want to do is to double click the file and play, you can't be bothered with the wrong ID3 tags. Yet you still want to see the Japanese text so that you can confirm your suspicion that the song is yet another by KOTOKO or any member of I've Sound.

Useful Tool 3: Unicode Rewriter

Some guy (much better at programming than me) is also pissed with the same problem (or maybe he's worse, he uses iTunes and iPod. Sad.). So he wrote a program that will convert the ID3 tags to Unicode, so that it will always show up correct in Windows XP and other Unicode-supporting OSes.

http://sourceforge.net/project/showfiles.php?group_id=113227

Download the installer unless you want to compile your own. Never mind that it's in .jar since it will work like the .exe installer. But you must have Java installed.

By default, if I'm not wrong, the selected input language is Chinese, but you can change it to Japanese Autodetect in the menu at the top left corner that pops up into more menues, 3 layers in total. Just drag and drop, and click Convert, done. Note that you may lose text that may not be translated to Unicode, and the file's time/datestamp will be changed.

However, this does not work for files that have been corrupted so bad that the ‘šŒŽ‚Í‚é‚© is not recognized as Japanese (but as Unicode instead). I'm still looking for a solution for this one (meaning force the text to be read as Japanese), so leave a message if you have the answer.

Situation 4: Playing The MP3 File In Windows Media Player (or your MP3 player) Shows ‘šŒŽ‚Í‚é‚©

Your Media Player or MP3 player may not support the Japanese/Chinese characters, or your MP3 player does not support Unicode (WMP supports Unicode).

While it's possible to get your media player to show the words correctly via AppLocale, it's stupid when all you do to play a file is to double-click on the file itself. Plus, you can't use AppLocale on your MP3 player.

Useful Tool 4: Foobar2000

So it's irritating that your MP3 player is displaying the Ë®˜ò¤«¤ª¤ê instead of the file name, which makes it very difficult to find the file you want. Also, certain MP3 players (i'm using one "PMP" which is like that) arrange songs alphabetically, yet displays the ID3 tag song name instead, so it becomes very messy.

To kill two birds with one stone again, Foobar2000 has a function called Remove Tags from File(s). (Drag the file into Foobar2000, right-click on the file, select Tagging -> Remove Tags from File(s)) It's good since it allows for mass de-tagging of files.

This time I won't post the link to Foobar2000. It's easy to find via google anyway, and it's free.

So there you have it, four situations that may piss you off (it did piss me off), and four solutions to the problems. Hope this article has been useful to you, and happy Otakuing!

Also, if there's anything more you need to ask, or if you want to add anything to the list, feel free to comment something. :)

Converting videos (including MKV RMVB etc) for watching on PSP - The Drag and Drop Approach

I've written something close to a guide on this once or twice, but they are scattered all over the place such that when people ask me, "How to convert MKV to watch on PSP?", I have nothing to answer them with. It'd be cool if I can tell them, "Just visit my blog". So here I am, trying to make it happen.

Yes, there are many many many many many all-in-one converters and PSP-specific converters out there, but why am I posting one that involves more than one program and is much more difficult to setup?

One reason is because I'm using this method. The other reason is that I was searching for a method to drag and drop any type of video file(AVI, MKV, RMVB, WMV, FLA (yes i mean youtube), and any other thing you can think of as long as it can be played on your system) and it will start converting, with the best quality possible. Now, you'd want to use my method, wouldn't you?

If put into a list, the pros of my method:

- Just drag and drop multiple files
Nothing else. Convenient and time-saving, something I required since I have lots to convert and little time with the com.

- Works with ANY (or close to that) video file format, including those with soft subtitles
This would be the most important reason you'd be viewing this guide. Heck, this is the reason why I'd even look into any method in the first place. Since I'm in the same shoes as you.

Anyway, the reason it works with the subtitles is because it involves using the only fast solution available, which is launching VSFilter(details below). You can use this with your own program to get the subtitles out. But most likely you'll be running into problems like not being able to read MKV or RMVB files or a/v sync problems with the said formats. So read on.

- Relatively fast
According to internet user comparisons, it's much faster than Video9, which can only match the speed after some tweaks that sacrifice quality (using 1-pass instead of 2, and the 1-pass isn't fully supported).

- Good quality
It encodes the video in H.264 unlike some that encodes in XviD because it's easier to play with the file format and getting the PSP to recognize the file, plus it's faster to encode in XviD. But H.264 produces much better quality for the same or lower file size.

Also, it uses Lanczos resize, one of the best resizing methods out there, to make the video have the same resolution as the PSP, eliminating the need for a second resizing by the PSP.

- And compared to some other (so-called "best") solutions that involves installing a homebrew player instead of using the built-in one:

When using the built-in player, the PSP uses the built-in hardware H.264 (and AAC?) acceleration for playback. When using a homebrew, it's "software" decoding. And the pros of the former are:

Higher bitrates allowed, meaning no lag even if you set the bitrate and quality high (way higher quality than XviD allows without lag when played with homebrew players). However, sometimes the file will be labelled as unsupported or corrupted when the quality is set higher, suggesting there is still a bitrate ceiling probably following the H.264 profiles.

VERY GOOD battery life, beating the crap out of IPod touch and many other MP3/MP4 players. See it to believe. Hint: at lowest brightness, it consumes slightly less than twice the power required for MP3 playback. Now, go find out how long the PSP lasts for MP3.

What about the cons?:

- They're at the bottom of the page (after it's finished)
So you'll have to read the rest of my guide first haha. Who would put the cons on their product packaging anyway?

How it works (a demostration)

1. Unzip and run 3GP_Converter.exe (first run may require selecting something... just select PSP)

2. Drag and drop your file inside the program. This is what you'll see. The file's name is in the white box. Choose the settings. It's at my settings for this example, but I also have other QScale (quality-based encoding) settings.


3. It starts encoding


4. As the first video is encoding you can drag more files inside for batch encode


5. Now you see two files. The MAQxxxxx.MP4 is your video. The file below with extension .THM is the thumbnail file for you to see the preview pictures on the PSP. It is not necessary for video playback, and its generation can be disabled.


Rename the files (to MAQxxxxx.MP4) for the PSP to arrange the video in ascending order. Rename the .THM file to the same name as the .MP4. Copy the files to Memory Stick:\MP_ROOT\101ANV01

6. Lets check out the results:


The subtitles are now hard-subbed!

7. Lets see how good the quality is. You know this scene is hard:


8. It's blurred because it's 480x272 upscaled to 2x. Originally you will see it as this:


9. More quality show-off:



OKAY SO HERE'S THE GUIDE:

Download the following files:

3GPConverter from Here

AVISynth 2.5 from Here

The problem with 3GPConverter is that if you search it in yahoo, you get a lot of different programs with the same name. This is the one we want. It's free, and all converters using free codecs/encoder engine or library should be. Yes, people are making money out of others' work.

Just extract the 3GPConverter from the archive. No installation required. Install AVISynth.

Now now, I'm supposed to write you a guide (on how to set up),

but why bother when I can give you the entire thing pre-set-up instead?

3GP_Converter034.zip

3GP_Converter034 Square.zip

Hmm File Den files are taken down, maybe I should consider using my Netfirms space.

The square version is for videos in the older 4:3 format.

Why use the square version when you can select 4:3 during playback, you may ask. Better quality and lower file size, sounds convincing enough to you?

Singapore Anime scene has no hope

...for a future... to be precise.

I reached home and switched on the TV. It was going to show Utawarerumono (after the usual 1.5-2 year lag after the first broadcast in Japan. At least this is a way big improvement from last time, so there's some hope here... or is there? Read on.). I was going to eat my dinner and there wasn't Futureweapons, Mythbusters, or any kind of sickingly funny video clips show on, my TV rig (as in computer) remained down as it had been for months, and I didn't have any Anime encoded onto DVD-Video. So I decided to watch, just the opening video.

As it faded in from black, I was like, "wth are those macroblocks? Yes they are microblocks." Ok, maybe that wasn't so bad since we're all familiar with Starhub Digital Cable's awesome quality. But then I thought I was also seeing mosquito noise in a majority of the scenes, and some amount of visible blocking. Then a whole screen of corrupted blocks, that or extremely bad macroblocking, slapped me in the face. That's definitely not something Starhub would give under normal circumstances. And towards the ending of the opening, "Please don't tell me that's stuttering due to bad 3:2 pulldown removal." The next scene told me that.

This is of great significance. Because, AFAIK, Anime is either 1) 23.976fps, 3:2 pulldowned to 29.97 and broadcasted, or 2) 23.976/29.97fps mixed, broadcasted as 29.97fps.

If it's case 1), the conversion that takes place would be 23.976fps -> 25fps which involves either increasing the speed of the video by 4% or the more complex method of 2:2:2:2:2:2:2:2:2:2:2:3 pulldown, both of which will not create stuttering THAT bad.

If it's case 2), which I believe to be the case (many recent Anime, I've discovered, have 29.97fps opening video), the higher source frame (and bigger difference in fps) makes less noticable motion artifacts (since there's more "data" for interpolation).

The point is not which case result in less stuttering, the point is that both approaches, if done correctly using the correct source, will produce less stuttering compared to what I saw. So, would it be case 3), or case 4)?

3) The source was 23.976fps, 3:2 pulldown-ed to 29.97fps, and converted directly to 25fps. This one results in a lot of noticable stuttering, since there is already some after the 3:2 pulldown.

4) The source was 29.97fps, 3:2 pulldown removal-ed 23.976 (noobishly), then converted to 25fps. This is the worst case scenario, since the most destruction happens in this case, when motion data in the 29.97fps video is thrown away. And also, this suggests a shady source (as in where the video came from is questionable, not that the source video is dark lol) - only fansubbers (or pirates) would be doing this kind of thing.

To clarify, more damage is done from improper 3:2 pulldown removal than frame rate conversion from 29.97fps to 25fps. This is because motion data is still kept via interpolation in the second case. However for the first, one entire frame is just thrown away like that. Imagine a person running at the 1m mark, then 2m, then 3m, 4m, and suddenly at 6m. Can you imagine the amount of stuttering? Yes it's the amount you're seeing in improper 3:2 pulldown removal-ed fansubs. Can you see where I'm getting at?

Yes, I've long suspected that the Anime shown on TV in Singapore are not original. This is just a new strong evidence to support.

But this isn't the sole reason why I say that Singapore Anime scene has no hope. Take a look a shops selling Anime figurines and other related stuff. They are closing one by one, even Takashimaya and Metro have stopped selling such stuff (or they shifted them to someplace I don't know). So much for the Odex saga; if so many people are watching Anime we should expect this business to flourish. And then there's Odex with their sad quality and slow releases (and the lack of many series). Plus, last time one can see Anime on channel 8 or central (or the late channel i) at hours other than midnight. Now it's all Cartoon Network and Nickelodeon shows during primetime, and Anime at late night hours on Arts Central. There really is a lack of exposure of Anime to the youngsters, something that can be bad since I know people who are still watching those kind of cartoons. While it does stall mental development, it also removes the supply of new Anime watchers and supporters, which will only decrease over time as we grow out of it. The reduction of the quality and number of Singaporean comic/manga artists publishing in newspapers is proof of this.

That's why I say we're doomed.

Da Capo II Anime: Afterthoughts

If I were to compile a list of disappointments in Anime/Manga for the year 2007 (maybe add 2008), this one is definitely going to be high up that chart.

Not that I didn't expect a disappointment (I knew something is wrong becuz it only had 13 eps compared to D.C.'s 26 with another 26 worth of epilogue), but I really didn't didn't didn't expect that much. In fact I was hoping for some explaination of the things that were exposed in the pre-main-story demo game. But what did they (the producers of the Anime) give us? A (possibly slightly altered) storyline based on the Minatsu route? Gimme a break. IMHO this is like nothing important compared to the rest of the story, but those who played D.C.II would know better.

List of disappointments (yes there is a whole list just in this Anime):

- Of all possible combinations, why Koko vs Minatsu???
- And even when so, Koko was a sad thing, Minatsu took all the limelight. Personally, I was against the huge Multi support in To Heart (including Remember My Memories), and is still very against android fetish if they are not the main character. This is no exception. Probably people feel sad when it comes to the coexistence of robots and humans, but this idea has been used too many times, and with geneboosted/genetically-modified humans, cyborgs, superhuman clones, man-made-human (meaning a non-robot body made totally from scratch, using magic or technology (or alchemy, which tends to be the case)), resurrected undead, monster half-blood, and summoned beings, why would a fucking robot with less processing power than a Pentium II cause such a controversy just because they are man-made and possess more ability than humans? (And lets not talk about witches/faeries/spirits/magic users in general here)
(Boy the above point is much longer than I expected)
- I want my Asakura sisters! I want my Oto-nee~! (Plus, the Asakura name means they have a lot to do with the story, I hope. But that's secondary. I want to be moe-ed to death by this kawaii sisters. Especially Oto-nee - most moe older sister, period.)
- Where is Nanaka? Isn't D.C.II called "The Nanaka Game"? Oh, episode 7. That's it? Hey duuuuuuuuudes, her grandmother created quite a problem in D.C. Anime even though she isn't part of the "true" storyline, and was the root of all problems in D.C.S.S.. Now her grandchild isn't even 10% as good. Gotta be kidding me.
- Suginami makes much less appearance much unlike the resourceful, sly, irritating but ultimately cool Suginami we know in D.C. and D.C.S.S. Anime, and in the D.C. series of games. (And a very good and very rare type of character he is)
- Where is the pain when people start to leave in the D.C. Anime (which I believe was a very good part by the director by making full use of the otherwise mediocre side storylines to fill up much of the latter 13 episodes with lots of story and sadness. If you noticed, most of the time a side storyline is only good for 2-4 episodes in Anime form. Planting a few of those in their most effective length made D.C. one of the most famous love Anime in ability to make people cry. And no I'm not talking about people who cry watching Gundam.) And where is the feeling of fukked-up-ness when Sakura couldn't control the magic Sakura tree and when you cannot decide who to sympathize with even when we all know Sakura is a selfish brat.
- Horie Yui and Tamura Yukari, the cute jie jie (meaning sister in Chinese) and loli goddess, didn't have enough screen time even though their characters are the most important in the story (i think)
- And most importantly, above all else,

(Spoiler warning) This Anime didn't explain the events after D.C., the immortality of Yoshino Sakura, the birth of Sakurai Yoshiyuki, the (ought to be) relationship between him and the Asakura sisters, and the root cause of all, the rebirth of the everlasting magic Sakura tree.

So, to conclude,

This is a plot by the producers to make us buy the game.

Actually I would've played the game before the Anime, if it ever gets translated.

And, this is one Anime that's not going to be remembered in Anime history, unlike its predecessors. The Kyoto/key series definitely did a better job at this.