Author Archives:

FaceBook Mutual Friend Scraper

Posted by vinaybalamuru on

I’ve been playing with RestFB lately. Here’s a simple program adapted from their demo code that extracts my friends ids and displays our mutual friends. Note that you will need to generate your own access token, either via your app or from the this link : https://developers.facebook.com/tools/explorer .

Clique analysis anyone?

package com.vgb.social.facebook;

import java.util.List;

import com.restfb.Connection;
import com.restfb.DefaultFacebookClient;
import com.restfb.FacebookClient;
import com.restfb.types.Page;
import com.restfb.types.Post;
import com.restfb.types.User;

public class FaceBookMutualFriendMiner {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		// TODO Auto-generated method stub

		String MY_ACCESS_TOKEN = "################# GET YOUR OWN ACCESS TOKEN #####################";

		FacebookClient facebookClient = new DefaultFacebookClient(MY_ACCESS_TOKEN);

		User user = facebookClient.fetchObject("me", User.class);

		System.out.println("User name: " + user.getName());
		Connection myFriends = facebookClient.fetchConnection("me/friends", User.class);
		Connection myFeed = facebookClient.fetchConnection("me/feed", Post.class);

		System.out.println("Count of my friends: " + myFriends.getData().size());

		processFriends(facebookClient, myFriends.getData(), true, "My Friend");

	}

	private static void processFriends(FacebookClient facebookClient, List myFriend, boolean recurse, String pre) {
		for (User u : myFriend) {
			System.out.println(pre + " : " + u.getName() + " " + u.getId());

			if (recurse) {
				Connection myMutualFriends = facebookClient.fetchConnection("me/mutualfriends/" + u.getId(), User.class);
				processFriends(facebookClient, myMutualFriends.getData(), false, "My Mutual Friends with " + u.getName());
			}
		}
	}

}

Basic Chunked List Iterator

Posted by vinaybalamuru on

Here’s a simple chunked iterator I wrote up that makes it easy to provide list data in chunks. This is useful when there is a need to break up and process large lists in pieces/chunks

    private static class ChunkList
            implements Iterable<List> {
        private final List list;
        private final int chunkSize;
        int cur;
        boolean hasNext = true;

        public ChunkList(List list, int chunkSize) {
            this.list = list;
            this.chunkSize = chunkSize;
            this.cur = 0;
        }

        @Override
        public Iterator<List> iterator() {
            return new Iterator<List>() {
                @Override
                public boolean hasNext() {
                    return hasNext;
                }

                @Override
                public List next() {
                    int start = cur;
                    int end = cur+chunkSize;
                    if (cur+chunkSize >= list.size()) {
                        end = list.size();
                        hasNext = false;
                    }

                    List subList =  list.subList(start, end);
                    cur = end;

                    return subList;
                }

                @Override
                public void remove() {
                    throw new UnsupportedOperationException("Chunk Removal not supported");
                }
            };
        }
    }

Test driver

    public static void main(String arg[]) {
        List list = new ArrayList();
        for (int i = 1; i             list.add(i);
        }

        ChunkList chunkList = new ChunkList(list, 10);

        Iterator<List> iter = chunkList.iterator();

        while (iter.hasNext()) {
            List chunk = iter.next();
            System.err.println("#### " + chunk);
        }
    }

Results

#### [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
#### [11, 12, 13, 14, 15, 16, 17, 18, 19, 20]
#### [21, 22, 23, 24]

Another (perhaps cleaner) alternative would have been to use a list iterator to “collect” elements withing the next() methods instead of employing sublist(…).

Enjoy.

Fix for Tomcat / SOLR Error java.net.SocketException: Connection reset by peer: socket write error

Posted by vinaybalamuru on

I came across this issue when trying to index data that contained a large number of literal key/values. By a process of elimination, I determined that Tomcat’s (6.x) maxHttpHeaderSize was being exceeded so I bumped it up a bit. Edit  the “Connector” element in <tomcat>/conf/server.xml as below. Hope this helps.

<Connector port="8080" protocol="HTTP/1.1"
 connectionTimeout="500000"
 redirectPort="8443"
 URIEncoding="UTF-8"
 maxPostSize="0"
 maxHttpHeaderSize="131072"
 />

SOLR 4 / MultiCore / Tomcat 6.x Install Simplified

Posted by vinaybalamuru on

For a few weeks now, I’ve been involved with getting SOLR up and running (actually a bit more than that :) . I’ve had my share of issues getting things to work exactly as intended and will be writing a series of posts that explain how to accomplish various tasks with this wonderful and powerful piece of software.

For starters, this post describes a simple way of getting multicore (multi-index for those of you used to lucene terminology) to work with a fresh tomcat install.

  • Download and install Tomcat 6.x in a directory of your choice. Tomcat 6.x can be downloaded from http://tomcat.apache.org/download-60.cgi
  • Download SOLR and unzip it to a directory, henceforth referred to as <solrdist>. We’re using SOLR 4 which is yet unreleased at the time of writng but can be downloaded from https://builds.apache.org/job/Solr-trunk/lastSuccessfulBuild/artifact/artifacts/
  • Create a directory that will host all the SOLR related files and directories, .. henceforth referred to <solrhost> (C:\java\solr4tomcatinstall\solrhost in this example).
  • Edit bin/catalina.bat by appending <solrhost> to the JAVA_OPT variable eg. add this line early in the file around where CATALINA_HOME is first referenced. Note that the Xmx parameter and value are optional (use / tweak this setting if you have heap  errors when using SOLR).

set JAVA_OPTS=%JAVA_OPTS% -Dsolr.solr.home=C:\java\solr4tomcatinstall\solrhost -Xmx1000m

  • Edit Tomcat’s conf/server.xml and add the following URIEncoding attribute to the correct Connector element:

<Server ...>
 <Service ...>
 <Connector ... URIEncoding="UTF-8"/>
 ...
 </Connector>
 </Service>
 </Server>

  • In the <solrhost> directory, create a file called solr.xml that looks like this.

<?xml version="1.0" encoding="UTF-8" ?>
 <solr persistent="true" sharedLib="lib">
 <cores adminPath="/admin/cores">
 <core schema="schema.xml" instanceDir="cores\" name="dummycore" dataDir="dummycore" conf="solrconfig.xml"/>
 </cores>
 </solr>

  • In the <solrhost> directory, create the following directories: cores, cores/conf and lib . The lib directory will contain all the shared SOLR and third party libs. The cores directory will contain the indices for all cores created in future
  • Now, copy all the jars (including those in subdirectories) from <solrdist>/contrib/* and <solrdist>/dist/ to <solrhost>/lib . These will be shared across all cores, as configured in the solr.xml a couple of steps above. Do not copy the solr*.war file.
  • Copy the solr*.war file from <solrdist>/dist/ to <tomcat>/webapps/ and rename it as solr.war .
  • Copy <solrdist>/ example/solr/conf to <solrhost>/cores/conf . The configuration will be customized later.
  • Now, attempt to start tomcat via <tomcat>/bin/catalina.bat run . There should be no errors
  • Navigate to the SOLR management UI at http://localhost:8080/solr/ . The UI should load and “dummycore” should be visible and available.

Tree Linux Command

Posted by vinaybalamuru on

Where have you been, all my life?

vinay@proton ~/workspaces/defaultworkspace/neo4j-tutorial $ tree  src/koan/java
src/koan/java
└── org
    └── neo4j
        └── tutorial
            ├── Koan01.java
            ├── Koan02.java
            ├── Koan03.java
            ├── Koan04.java
            ├── Koan05.java
            ├── Koan06.java
            ├── Koan07.java
            ├── Koan08a.java
            ├── Koan08b.java
            ├── Koan08c.java
            ├── Koan08d.java
            ├── Koan08e.java
            ├── Koan09.java
            ├── Koan10.java
            ├── Koan11.java
            ├── koan12
            │   └── HomePlanetUnmanagedExtension.java
            ├── Koan12.java
            ├── koan13
            │   └── AwesomenessServerPlugin.java
            ├── Koan13.java
            ├── koan14
            │   ├── SalaryUnmanagedExtension.java
            │   └── UserNameAndPasswordForSalariesSecurityRule.java
            ├── Koan14.java
            ├── matchers
            │   ├── CharacterAutoIndexContainsSpecificCharacters.java
            │   ├── ContainsOnlyHumanCompanions.java
            │   ├── ContainsOnlySpecificActors.java
            │   ├── ContainsOnlySpecificInts.java
            │   ├── ContainsOnlySpecificNodes.java
            │   ├── ContainsOnlySpecificSpecies.java
            │   ├── ContainsOnlySpecificStrings.java
            │   ├── ContainsOnlySpecificTitles.java
            │   ├── ContainsSpecificCompanions.java
            │   ├── ContainsSpecificNumberOfNodes.java
            │   ├── ContainsWikipediaEntries.java
            │   └── PathsMatcher.java
            └── server
                └── rest
                    └── domain
                        ├── EpisodeSearchResult.java
                        └── EpisodeSearchResults.java

10 directories, 36 files